Boost C++ Libraries: Ticket #12398: Boost log writes much less records in remote disk than local disk

status changed; resolution set

Andrey Semashev — Wed, 17 Aug 2016 22:01:55 GMT

status new → closed
resolution → invalid

Boost.Log has no knowledge about the nature of the filesystem, it works the same regardless. If your sinks are asynchronous then make sure log records are not piling up in the queue or being dropped due to the queue capacity limit. Try switching to synchronous logging or a non-dropping queueing policy and make sure you flush the sink before the application exit. Also check that you're using the same filter in both cases.

xinyan.zhang@… — Thu, 18 Aug 2016 19:07:56 GMT

My code uses asynchronous sink without setting any queuing policy. Does that mean "unbounded_fifo_queue" is used by default? And the queue has unlimited depth? If so, it could grow to whatever within the memory size. In my machine the maximum memory usage is less than 70%. The application itself uses about 800 MB memory. I did see any memory over flow.

xinyan.zhang@… — Fri, 19 Aug 2016 21:24:47 GMT

Sorry, the last sentence in previous reply should be " I did NOT see any memory over flow. "

Is there any way to check if messages are dropped or not? My way is: I added a counter, every time it increased by +1 when writing a log record. Then after the test, I waited for 20 minutes to ensure log records were written into the file. And then shut down the application. I found last counter is 10,000 larger than the total line number. In other words, almost 10,000 lines of record are missing in the log file.

Andrey Semashev — Sat, 20 Aug 2016 00:36:11 GMT

Is there any way to check if messages are dropped or not?

There is not. Unless there is a bug or memory overflow, unbounded_fifo_queue should not drop records. You'll have to debug the problem yourself. I'd start by counting the number of log records that passed the filter (asynchronous_sink::consume) and also the number of records that reached the file stream (text_file_backend::consume). Also try switching to synchronous_sink.

xinyan.zhang@… — Thu, 25 Aug 2016 21:06:57 GMT

I found the reason why less log records are in file. It is not related to the boost library, but our test tool which has problems. So, less log records are generated. Thank you very much.