Skip to content

main: log file #2748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
Aug 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
62de8a6
initial, base LOG macro
staviq Aug 23, 2023
727af3e
add *.log to .gitignore
staviq Aug 23, 2023
d5156d3
added basic log file handler
staviq Aug 23, 2023
356a166
reverted log auto endline to better mimic printf
staviq Aug 23, 2023
71d05b9
remove atomics and add dynamic log target
staviq Aug 23, 2023
47b9f2d
log_enable/disable, LOG_TEE, basic usage doc
staviq Aug 24, 2023
f5080da
update .gitignore
staviq Aug 24, 2023
8054c32
Merge branch 'master' into betterlogs
staviq Aug 24, 2023
45f1a43
mv include to common, params, help msg
staviq Aug 24, 2023
360b36c
log tostring helpers, token vectors pretty prints
staviq Aug 25, 2023
181e8a9
main: replaced fprintf/LOG_TEE, some trace logging
staviq Aug 25, 2023
54e81ba
Merge branch 'ggerganov:master' into betterlogs
staviq Aug 25, 2023
4fdcede
LOG_DISABLE_LOGS compile flag, wrapped f in macros
staviq Aug 25, 2023
c8a1118
fix LOG_TEELN and configchecker
staviq Aug 25, 2023
9bace22
stub LOG_DUMP_CMDLINE for WIN32 for now
staviq Aug 25, 2023
0b36725
fix msvc
staviq Aug 26, 2023
e99f039
cleanup main.cpp:273
staviq Aug 26, 2023
5031c50
Merge branch 'master' into betterlogs
staviq Aug 26, 2023
5cea869
fix stray whitespace after master sync
staviq Aug 26, 2023
2c1930d
Merge branch 'master' into HEAD
ggerganov Aug 29, 2023
b97958a
log : fix compile warnings
ggerganov Aug 29, 2023
6b4c65b
log : do not append to existing log + disable file line func by default
ggerganov Aug 29, 2023
891ac40
Merge branch 'master' into HEAD
ggerganov Aug 29, 2023
5c978f4
log : try to fix Windows build
ggerganov Aug 29, 2023
c72d344
main : wip logs
ggerganov Aug 29, 2023
ecdf113
main : add trace log
ggerganov Aug 29, 2023
f7ac431
Merge branch 'ggerganov:master' into betterlogs
staviq Aug 29, 2023
3a10f5a
review: macro f lowercase, str append to sstream
staviq Aug 29, 2023
ba5590f
review: simplify ifs and str comparisons
staviq Aug 29, 2023
3edee3f
fix MSVC, formatting, FMT/VAL placeholders
staviq Aug 29, 2023
6fa208e
review: if/else cleanup
staviq Aug 29, 2023
f60f7d3
review: if/else cleanup (2)
staviq Aug 29, 2023
89dc100
replace _ prefix with _impl suffix
staviq Aug 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
*.bin
*.exe
*.dll
*.log
.DS_Store
.build/
.cache/
Expand Down
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,11 @@ k_quants.o: k_quants.c k_quants.h
$(CC) $(CFLAGS) -c $< -o $@
endif # LLAMA_NO_K_QUANTS

ifdef LLAMA_DISABLE_LOGS
CFLAGS += -DLOG_DISABLE_LOGS
CXXFLAGS += -DLOG_DISABLE_LOGS
endif # LLAMA_DISABLE_LOGS

#
# Print build information
#
Expand Down Expand Up @@ -356,7 +361,7 @@ OBJS += ggml-alloc.o
llama.o: llama.cpp ggml.h ggml-alloc.h ggml-cuda.h ggml-metal.h llama.h
$(CXX) $(CXXFLAGS) -c $< -o $@

common.o: common/common.cpp common/common.h build-info.h
common.o: common/common.cpp common/common.h build-info.h common/log.h
$(CXX) $(CXXFLAGS) -c $< -o $@

console.o: common/console.cpp common/console.h
Expand Down
22 changes: 22 additions & 0 deletions common/common.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,9 @@ bool gpt_params_parse(int argc, char ** argv, gpt_params & params) {
}
} else if (arg == "-h" || arg == "--help") {
gpt_print_usage(argc, argv, default_params);
#ifndef LOG_DISABLE_LOGS
log_print_usage();
#endif // LOG_DISABLE_LOGS
exit(0);
} else if (arg == "--random-prompt") {
params.random_prompt = true;
Expand Down Expand Up @@ -519,6 +522,25 @@ bool gpt_params_parse(int argc, char ** argv, gpt_params & params) {
std::istreambuf_iterator<char>(),
std::back_inserter(params.grammar)
);
#ifndef LOG_DISABLE_LOGS
// Parse args for logging parameters
} else if ( log_param_single_parse( argv[i] ) ) {
// Do nothing, log_param_single_parse automatically does it's thing
// and returns if a match was found and parsed.
} else if ( log_param_pair_parse( /*check_but_dont_parse*/ true, argv[i] ) ) {
// We have a matching known parameter requiring an argument,
// now we need to check if there is anything after this argv
// and flag invalid_param or parse it.
if (++i >= argc) {
invalid_param = true;
break;
}
if( !log_param_pair_parse( /*check_but_dont_parse*/ false, argv[i-1], argv[i]) ) {
invalid_param = true;
break;
}
// End of Parse args for logging parameters
#endif // LOG_DISABLE_LOGS
} else {
fprintf(stderr, "error: unknown argument: %s\n", arg.c_str());
gpt_print_usage(argc, argv, default_params);
Expand Down
3 changes: 3 additions & 0 deletions common/common.h
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

#include "llama.h"

#define LOG_NO_FILE_LINE_FUNCTION
#include "log.h"

#include <string>
#include <vector>
#include <random>
Expand Down
Loading