Skip to content

Fix bugs related to UTF-8 #604

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 7, 2025
Merged

Fix bugs related to UTF-8 #604

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c4aed8e
fix: data initialization when utf-8 mode is enabled
yutaro-sakamoto Mar 5, 2025
0a2b069
fix: display statement for utf-8 mode
yutaro-sakamoto Mar 5, 2025
43f5833
test: add test for changes
yutaro-sakamoto Mar 5, 2025
3dd4a4e
test: fix new tests
yutaro-sakamoto Mar 5, 2025
371a585
chore: improve #if macros
yutaro-sakamoto Mar 5, 2025
7ed06b7
fix: revert cobj/pplex.c
yutaro-sakamoto Mar 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cobj/codegen.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2452,6 +2452,7 @@ static void joutput_initialize_one(struct cb_initialize *p, cb_tree x) {
joutput_data(x);
joutput(".fillBytes(%d, %d);\n", buffchar, f->size);
} else {
#if !I18N_UTF8
if (f->size >= 8) {
buffchar = *(buff + f->size - 1);
int n = 0;
Expand All @@ -2471,6 +2472,7 @@ static void joutput_initialize_one(struct cb_initialize *p, cb_tree x) {
return;
}
}
#endif
joutput_data(x);
#if I18N_UTF8
joutput(".setByByteArrayAndPaddingSpaces (");
Expand Down
4 changes: 1 addition & 3 deletions libcobj/app/src/main/java/jp/osscons/opensourcecobol/libcobj/termio/CobolTerminal.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,7 @@ private static void displayAlnum(AbstractCobolField f, PrintStream stream) {
CobolDataStorage storage = f.getDataStorage();
if (CobolUtil.terminalEncoding == CobolEncoding.UTF8) {
byte[] utf8Bytes =
new String(
storage.getByteArrayRef(0, f.getSize()),
AbstractCobolField.charSetSJIS)
new String(storage.getByteArray(0, f.getSize()), AbstractCobolField.charSetSJIS)
.getBytes(StandardCharsets.UTF_8);
stream.write(utf8Bytes, 0, utf8Bytes.length);
} else {
Expand Down
53 changes: 53 additions & 0 deletions tests/cobol_utf8.src/pic-x.at
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -561,6 +561,59 @@ AT_CHECK([diff out1.txt out2.txt])

AT_CLEANUP

AT_SETUP([Group item COB_TERMINAL_ENCODING])

AT_DATA([prog.cbl], [
IDENTIFICATION DIVISION.
PROGRAM-ID. prog.
DATA DIVISION.
WORKING-STORAGE SECTION.
01 WK-AREA.
03 WK-CD PIC X(04).
03 WK-NAME PIC X(20).
03 WK-DPT-CD PIC X(02).
03 WK-ENT-DATE PIC 9(08).
03 WK-RETURN PIC 9(01).
PROCEDURE DIVISION.
MAIN.
MOVE "0012千葉 二郎 02199904010" TO WK-AREA.
DISPLAY "従業員番号: " WK-CD.
DISPLAY "名前　　　: " WK-NAME.
DISPLAY "部署コード: " WK-DPT-CD.
DISPLAY "入社日　　: " WK-ENT-DATE.
STOP RUN.
])

AT_CHECK([cobj prog.cbl])
AT_CHECK([COB_TERMINAL_ENCODING=UTF-8 java prog], [0],
[従業員番号: 0012
名前　　　: 千葉 二郎 @&t@
部署コード: 02
入社日　　: 19990401
])

AT_CLEANUP

AT_SETUP([Initializing Group items])

AT_DATA([prog.cbl], [
IDENTIFICATION DIVISION.
PROGRAM-ID. prog.
DATA DIVISION.
WORKING-STORAGE SECTION.
01 F0 PIC X(17) VALUE "千葉 二郎".
PROCEDURE DIVISION.
DISPLAY F0.
STOP RUN.
])

AT_CHECK([cobj prog.cbl])
AT_CHECK([COB_TERMINAL_ENCODING=UTF-8 java prog], [0],
[千葉 二郎 @&t@
])

AT_CLEANUP

#AT_SETUP([Readable string literals])
#export LC_ALL=''
## Older compilers converts string literals "日本語" in COBOL source code
Expand Down
Loading