log library extensions

[vgrinenko@…]

Please forward this request to Mr. Andrey Semashev.

Dear Andrey,

in our company we have decided to use the boost log in our applications. We need 2 additional extensions for the file backend:

1. we want to delete the log files after specific amount of time
2. we want to make some logs un-deleteable, for example to change the log name in some circumstance, for example in case if the record with severity “fatal” has been written.

For this I have added to your text_file_backend.hpp/ text_file_backend.cpp two additional predicates and some code for these predicates support.

The time_based_deleting_predicate in my application is implemented as:

m_sink->locked_backend()->set_file_collector(sinks::polytec_ext::file::make_collector(
				keywords::target = dir
				, keywords::max_size = maxSize
				, keywords::time_based_deleting = [=](std::time_t time)
			{
				if (deletingInterval.count() == 0)
					return false;
				std::chrono::system_clock::time_point now = std::chrono::system_clock::now();
				std::time_t latest = std::chrono::system_clock::to_time_t(now) - deletingInterval.count();
				return latest > time;
			}));

The record_based_renaming_predicate is implemented as:

			m_sink->locked_backend()->set_record_based_renaming(
				[=](logging::record_view const& rec, boost::filesystem::path& path)
			{
				auto level = boost::log::extract< severity_level >("Severity", rec);
				if (level && severity_level::fatal == level.get())
				{
					auto fname = path.stem();
					bool hasErrorSuffix = boost::algorithm::ends_with(fname.c_str(), L"_ERROR");
					if (!hasErrorSuffix)
					{
						auto newFname = path.parent_path() /
							boost::filesystem::path(path.stem().wstring() + L"_ERROR" + path.extension().wstring());
						path = newFname;
						return true;
					}
				}
				return false;
			});

If you are finding these features useful for other library users and merging (may be with some improvements accordingly to your vision of the library future) with the library code it will make me a pleasure :-) and we will remove our implementation and will use the library “as is”. The ZIP file with my hpp an cpp is attached to the request. I have used 1.57 vesrsion of boost. Thank you for your attention and best regards. Victor Grinenko.

[Andrey Semashev]

Thanks for the proposal.

The time_based_deleting feature looks interesting, although I think it would be easier to use if it was just a max age specification instead of a predicate.

The record-based renaming seems too specific to me, and potentially complicated. It seems possible for the rename to actually move the file to another location, which would potentially require a copy operation. Also, in the current implementation I try to minimize the number of file operations because each of them is a potential point of failure. I don't think this feature is useful enough in general to warrant the risks. You can always have that kind of functionality by post-processing log files.

[vgrinenko@…]

a max age specification instead of a predicate.

I was trying to do similar to your implementation and more or less “generic”: max age or older than specific time or something else.

The record-based renaming seems too specific to me...

I agree, probably it would be better provide “generic file manipulation” possibility instead of renaming and allow to do something with m_File in the predicate, for example copy, move, close and open new log dependently from the record content. In this case the predicate itself will be responsible for the implementation and robustness and risks, not a library. But it is just an idea.

Thanks for the answer and good luck!

[anonymous]

Hi Andrey,

I have a problem with MFC applications, both shared and static. If logger is called from the std::thread there are no problems, but if from the concurrency::create_task - there are lot of memory leaks. Probably they are "fake" leaks, it happens if shared MFC unloaded too early, but the problem exist with a static MFC as well. Do you have any ideas?

To reproduce the problem create an empty console application project with MFC, add code below and set paths to boost include and libs.

Best regards, Victor.

// ConsoleApplicationMFC.cpp : Defines the entry point for the console application.
//

#define _CRT_SECURE_NO_WARNINGS
#define BOOST_SYSTEM_NO_DEPRECATED
#define BOOST_LIB_DIAGNOSTIC
#define CGAL_LIB_DIAGNOSTIC

// From StdAfx.h ->
#include <SDKDDKVer.h>

#include <stdio.h>
#include <tchar.h>
#define _ATL_CSTRING_EXPLICIT_CONSTRUCTORS      // some CString constructors will be explicit
#define _AFX_NO_MFC_CONTROLS_IN_DIALOGS         // remove support for MFC controls in dialogs

#ifndef VC_EXTRALEAN
#define VC_EXTRALEAN            // Exclude rarely-used stuff from Windows headers
#endif

#include <afx.h>
#include <afxwin.h>         // MFC core and standard components
#include <afxext.h>         // MFC extensions
#ifndef _AFX_NO_OLE_SUPPORT
#include <afxdtctl.h>           // MFC support for Internet Explorer 4 Common Controls
#endif
#ifndef _AFX_NO_AFXCMN_SUPPORT
#include <afxcmn.h>                     // MFC support for Windows Common Controls
#endif // _AFX_NO_AFXCMN_SUPPORT

#include <iostream>

// <- From StdAfx.h

#include <boost/log/core.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/attributes.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <boost/log/sources/severity_channel_logger.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/support/date_time.hpp>
#include <boost/log/utility/setup/console.hpp>
#include <boost\log\sinks\text_file_backend.hpp>

#if !defined(BOOST_LOG_NO_THREADS)
#include <boost/thread/locks.hpp>
#include <boost/thread/mutex.hpp>
#endif // !defined(BOOST_LOG_NO_THREADS)
#include <ppltasks.h>
#include <thread>


#ifdef _DEBUG
#define new DEBUG_NEW
#endif


// The one and only application object

CWinApp theApp;

using namespace std;
namespace logging = boost::log;
namespace src = boost::log::sources;
namespace sinks = boost::log::sinks;
namespace keywords = boost::log::keywords;
namespace expr = boost::log::expressions;

BOOST_LOG_ATTRIBUTE_KEYWORD(a_channel, "Channel", std::string)

int main()
{
	int nRetCode = 0;

	HMODULE hModule = ::GetModuleHandle(nullptr);

	if (hModule != nullptr)
	{
		// initialize MFC and print and error on failure
		if (!AfxWinInit(hModule, nullptr, ::GetCommandLine(), 0))
		{
			// TODO: change error code to suit your needs
			wprintf(L"Fatal Error: MFC initialization failed\n");
			nRetCode = 1;
		}
		else
		{
			// TODO: code your application's behavior here.

			typedef sinks::synchronous_sink<sinks::text_file_backend> file_sink;

			src::severity_channel_logger_mt<boost::log::trivial::severity_level> logger(keywords::channel = "L");
			boost::shared_ptr<file_sink> sinkT(new file_sink(keywords::file_name = "logs\\TestLogger_%6N.log"));
			sinkT->set_formatter
			(
				expr::stream
				<< " TimeStamp: " << expr::format_date_time< boost::posix_time::ptime >("TimeStamp", "%Y-%m-%d %H:%M:%S")
				<< " ThreadID: " << expr::attr<logging::thread_id>("ThreadID")
				<< " Message: " << expr::smessage
			);
			logging::core::get()->add_sink(sinkT);
			sinkT->set_filter(a_channel == "L");
			logging::add_common_attributes();

			BOOST_LOG(logger) << L"A message from the MAIN thread";

			std::thread testTthread([&]()
			{
				BOOST_LOG(logger) << L"A message from the std::thread";
			});
			testTthread.join();

			auto task = concurrency::create_task([&]()
			{
				// Produces memory leaks
				BOOST_LOG(logger) << L"A message from the concurrency::create_task thread";
			});
			
			task.wait();
			BOOST_LOG(logger) << L"Is concurrency::create_task done: " << task.is_done();
		}
	}
	else
	{
		// TODO: change error code to suit your needs
		wprintf(L"Fatal Error: GetModuleHandle failed\n");
		nRetCode = 1;
	}

	return nRetCode;
}

[vgrinenko@…]

Hi, it has nothing with MFC, sorry. Just this code is enough.

Visual Studio 2015.

For example:

Detected memory leaks!
Dumping objects ->
{887} normal block at 0x00B6A0F8, 128 bytes long.
 Data: < TimeStamp: 2017> 00 54 69 6D 65 53 74 61 6D 70 3A 20 32 30 31 37 
...

// ConsoleApplication_wo_MFC.cpp : Defines the entry point for the console application.
//

#define _CRT_SECURE_NO_WARNINGS
#define BOOST_SYSTEM_NO_DEPRECATED
#define BOOST_LIB_DIAGNOSTIC
#define CGAL_LIB_DIAGNOSTIC

#include <stdio.h>
#include <tchar.h>
#include <windows.h>
#include <ppltasks.h>
#include <thread>

#define _CRTDBG_MAP_ALLOC
#include <stdlib.h>
#include <crtdbg.h>

#include <boost/log/core.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/attributes.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <boost/log/sources/severity_channel_logger.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/support/date_time.hpp>
#include <boost/log/utility/setup/console.hpp>
#include <boost\log\sinks\text_file_backend.hpp>
#include <boost/locale/localization_backend.hpp>

#if !defined(BOOST_LOG_NO_THREADS)
#include <boost/thread/locks.hpp>
#include <boost/thread/mutex.hpp>
#endif // !defined(BOOST_LOG_NO_THREADS)

using namespace std;
namespace logging = boost::log;
namespace src = boost::log::sources;
namespace sinks = boost::log::sinks;
namespace keywords = boost::log::keywords;
namespace expr = boost::log::expressions;

BOOST_LOG_ATTRIBUTE_KEYWORD(a_channel, "Channel", std::string)

int main()
{
	int nRetCode = 0;
	_CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF);

	typedef sinks::synchronous_sink<sinks::text_file_backend> file_sink;

	src::severity_channel_logger_mt<boost::log::trivial::severity_level> logger(keywords::channel = "L");
	boost::shared_ptr<file_sink> sinkT(new file_sink(keywords::file_name = "logs\\TestLogger_%6N.log"));
	sinkT->set_formatter
	(
		expr::stream
		<< " TimeStamp: " << expr::format_date_time< boost::posix_time::ptime >("TimeStamp", "%Y-%m-%d %H:%M:%S")
		<< " ThreadID: " << expr::attr<logging::thread_id>("ThreadID")
		<< " Message: " << expr::smessage
	);
	logging::core::get()->add_sink(sinkT);
	sinkT->set_filter(a_channel == "L");
	logging::add_common_attributes();

	BOOST_LOG(logger) << L"A message from the MAIN thread";

	std::thread testTthread([&]()
	{
		BOOST_LOG(logger) << L"A message from the std::thread";
	});
	testTthread.join();

	auto task = concurrency::create_task([&]()
	{
		// Produces memory leaks
		BOOST_LOG(logger) << L"A message from the concurrency::create_task thread";
	});

	task.wait();
	BOOST_LOG(logger) << L"Is concurrency::create_task done: " << task.is_done();
	return nRetCode;
}

[vgrinenko@…]

Boost versions: 1.57 and 1.65.1