RDF progress helper.

This class provides callback functions to the RDataFrame. The event statistics (including elapsed time, currently processed file, currently processed events, the rate of event processing and an estimated remaining time (per file being processed)) are recorded and printed in the terminal every m events and every n seconds. ProgressHelper::operator()(unsigned int, T&) is thread safe, and can be used as a callback in MT mode. ProgressBar should be added after creating the dataframe object (df):

ROOT::RDataFrame df("tree", "file.root");

ROOT::RDF::Experimental::AddProgressBar(df);

ROOT::RDataFrame

ROOT's RDataFrame offers a modern, high-level interface for analysis of data stored in TTree ,...

Definition RDataFrame.hxx:41

ROOT::RDF::Experimental::AddProgressBar

void AddProgressBar(ROOT::RDF::RNode df)

Add ProgressBar to a ROOT::RDF::RNode.

Definition RDFHelpers.cxx:375

alternatively RDataFrame can be cast to an RNode first giving it more flexibility. For example, it can be called at any computational node, such as Filter or Define, not only the head node, with no change to the ProgressBar function itself:

ROOT::RDataFrame df("tree", "file.root");
auto df_1 = ROOT::RDF::RNode(df.Filter("x>1"));
ROOT::RDF::Experimental::AddProgressBar(df_1);

Definition at line 325 of file RDFHelpers.hxx.

Public Member Functions
	ProgressHelper (std::size_t increment, unsigned int totalFiles=1, unsigned int progressBarWidth=40, unsigned int printInterval=1, bool useColors=true)
	Create a progress helper.

	~ProgressHelper ()=default

unsigned int	ComputeCurrentFileIdx () const

std::size_t	ComputeNEventsSoFar () const

template<typename T >
void	operator() (T &)
	Thread-safe callback for RDataFrame.

template<typename T >
void	operator() (unsigned int, T &value)
	Thread-safe callback for RDataFrame.

void	registerNewSample (unsigned int, const ROOT::RDF::RSampleInfo &id)
	Register a new sample for completion statistics.

Private Member Functions
double	EvtPerSec () const
	Compute a running mean of events/s.

void	PrintProgressBar (std::ostream &stream, std::size_t currentEventCount) const
	Print a progress bar of width `ProgressHelper::fBarWidth` if `fGetNEventsOfCurrentFile` is known.

void	PrintStats (std::ostream &stream, std::size_t currentEventCount, std::chrono::seconds totalElapsedSeconds) const
	Print event and time statistics.

void	PrintStatsFinal (std::ostream &stream, std::chrono::seconds totalElapsedSeconds) const

std::pair< std::size_t, std::chrono::seconds >	RecordEvtCountAndTime ()
	Record current event counts and time stamp, populate evts/s statistics array.

Private Attributes
unsigned int	fBarWidth

std::chrono::time_point< std::chrono::system_clock >	fBeginTime = std::chrono::system_clock::now()

std::array< double, 20 >	fEventsPerSecondStatistics

std::size_t	fEventsPerSecondStatisticsIndex {0}

std::size_t	fIncrement

bool	fIsTTY

std::chrono::time_point< std::chrono::system_clock >	fLastPrintTime = fBeginTime

std::size_t	fLastProcessedEvents {0}

std::chrono::seconds	fPrintInterval {1}

std::mutex	fPrintMutex

std::atomic< std::size_t >	fProcessedEvents {0}

std::map< std::string, ULong64_t >	fSampleNameToEventEntries

std::mutex	fSampleNameToEventEntriesMutex

unsigned int	fTotalFiles

std::shared_ptr< TTree >	fTree {nullptr}

bool	fUseShellColours

Friends
class	ProgressBarAction

#include <ROOT/RDFHelpers.hxx>

Constructor & Destructor Documentation

◆ ProgressHelper()

ROOT::RDF::Experimental::ProgressHelper::ProgressHelper	(	std::size_t	increment,
		unsigned int	totalFiles = 1,
		unsigned int	progressBarWidth = 40,
		unsigned int	printInterval = 1,
		bool	useColors = true )

Create a progress helper.

Parameters

increment	RDF callbacks are called every `n` events. Pass this `n` here.
totalFiles	read total number of files in the RDF.
progressBarWidth	Number of characters the progress bar will occupy.
printInterval	Update every stats every `n` seconds.
useColors	Use shell colour codes to colour the output. Automatically disabled when we are not writing to a tty.

Definition at line 147 of file RDFHelpers.cxx.

◆ ~ProgressHelper()

ROOT::RDF::Experimental::ProgressHelper::~ProgressHelper ( )

default

Member Function Documentation

◆ ComputeCurrentFileIdx()

unsigned int ROOT::RDF::Experimental::ProgressHelper::ComputeCurrentFileIdx ( ) const

inline

Definition at line 448 of file RDFHelpers.hxx.

◆ ComputeNEventsSoFar()

std::size_t ROOT::RDF::Experimental::ProgressHelper::ComputeNEventsSoFar ( ) const

inline

Definition at line 439 of file RDFHelpers.hxx.

◆ EvtPerSec()

double ROOT::RDF::Experimental::ProgressHelper::EvtPerSec ( ) const

private

Compute a running mean of events/s.

Definition at line 164 of file RDFHelpers.cxx.

◆ operator()() [1/2]

template<typename T >

void ROOT::RDF::Experimental::ProgressHelper::operator() ( T & )

inline

Thread-safe callback for RDataFrame.

It will record elapsed times and event statistics, and print a progress bar every n seconds (set by the fPrintInterval).

Parameters

value Ignored.

Definition at line 403 of file RDFHelpers.hxx.

◆ operator()() [2/2]

template<typename T >

void ROOT::RDF::Experimental::ProgressHelper::operator()	(	unsigned int	,
		T &	value )

inline

Thread-safe callback for RDataFrame.

It will record elapsed times and event statistics, and print a progress bar every n seconds (set by the fPrintInterval).

Parameters

slot	Ignored.
value	Ignored.

Definition at line 393 of file RDFHelpers.hxx.

◆ PrintProgressBar()

void ROOT::RDF::Experimental::ProgressHelper::PrintProgressBar	(	std::ostream &	stream,
		std::size_t	currentEventCount ) const

private

Print a progress bar of width ProgressHelper::fBarWidth if fGetNEventsOfCurrentFile is known.

Definition at line 305 of file RDFHelpers.cxx.

◆ PrintStats()

void ROOT::RDF::Experimental::ProgressHelper::PrintStats	(	std::ostream &	stream,
		std::size_t	currentEventCount,
		std::chrono::seconds	totalElapsedSeconds ) const

private

Print event and time statistics.

Definition at line 227 of file RDFHelpers.cxx.

◆ PrintStatsFinal()

void ROOT::RDF::Experimental::ProgressHelper::PrintStatsFinal	(	std::ostream &	stream,
		std::chrono::seconds	totalElapsedSeconds ) const

private

Definition at line 275 of file RDFHelpers.cxx.

◆ RecordEvtCountAndTime()

std::pair< std::size_t, std::chrono::seconds > ROOT::RDF::Experimental::ProgressHelper::RecordEvtCountAndTime ( )

private

Record current event counts and time stamp, populate evts/s statistics array.

Definition at line 176 of file RDFHelpers.cxx.

◆ registerNewSample()

void ROOT::RDF::Experimental::ProgressHelper::registerNewSample	(	unsigned int	,
		const ROOT::RDF::RSampleInfo &	id )

inline

Register a new sample for completion statistics.

See also: ROOT::RDF::RInterface::DefinePerSample(). The id.AsString() refers to the name of the currently processed file. The idea is to populate the event entries in the fSampleNameToEventEntries map by selecting the greater of the two values: id.EntryRange().second which is the upper event entry range of the processed sample and the current value of the event entries in the fSampleNameToEventEntries map. In the single threaded case, the two numbers are the same as the entry range corresponds to the number of events in an individual file (each sample is simply a single file). In the multithreaded case, the idea is to accumulate the higher event entry value until the total number of events in a given file is reached.

Definition at line 382 of file RDFHelpers.hxx.

Friends And Related Symbol Documentation

◆ ProgressBarAction

friend class ProgressBarAction

friend

Definition at line 369 of file RDFHelpers.hxx.

Member Data Documentation

◆ fBarWidth

unsigned int ROOT::RDF::Experimental::ProgressHelper::fBarWidth

private

Definition at line 347 of file RDFHelpers.hxx.

◆ fBeginTime

std::chrono::time_point<std::chrono::system_clock> ROOT::RDF::Experimental::ProgressHelper::fBeginTime = std::chrono::system_clock::now()

private

Definition at line 333 of file RDFHelpers.hxx.

◆ fEventsPerSecondStatistics

std::array<double, 20> ROOT::RDF::Experimental::ProgressHelper::fEventsPerSecondStatistics

private

Definition at line 344 of file RDFHelpers.hxx.

◆ fEventsPerSecondStatisticsIndex

std::size_t ROOT::RDF::Experimental::ProgressHelper::fEventsPerSecondStatisticsIndex {0}

private

Definition at line 345 of file RDFHelpers.hxx.

◆ fIncrement

std::size_t ROOT::RDF::Experimental::ProgressHelper::fIncrement

private

Definition at line 339 of file RDFHelpers.hxx.

◆ fIsTTY

bool ROOT::RDF::Experimental::ProgressHelper::fIsTTY

private

Definition at line 351 of file RDFHelpers.hxx.

◆ fLastPrintTime

std::chrono::time_point<std::chrono::system_clock> ROOT::RDF::Experimental::ProgressHelper::fLastPrintTime = fBeginTime

private

Definition at line 334 of file RDFHelpers.hxx.

◆ fLastProcessedEvents

std::size_t ROOT::RDF::Experimental::ProgressHelper::fLastProcessedEvents {0}

private

Definition at line 338 of file RDFHelpers.hxx.

◆ fPrintInterval

std::chrono::seconds ROOT::RDF::Experimental::ProgressHelper::fPrintInterval {1}

private

Definition at line 335 of file RDFHelpers.hxx.

◆ fPrintMutex

std::mutex ROOT::RDF::Experimental::ProgressHelper::fPrintMutex

private

Definition at line 350 of file RDFHelpers.hxx.

◆ fProcessedEvents

std::atomic<std::size_t> ROOT::RDF::Experimental::ProgressHelper::fProcessedEvents {0}

private

Definition at line 337 of file RDFHelpers.hxx.

◆ fSampleNameToEventEntries

std::map<std::string, ULong64_t> ROOT::RDF::Experimental::ProgressHelper::fSampleNameToEventEntries

private

Definition at line 342 of file RDFHelpers.hxx.

◆ fSampleNameToEventEntriesMutex

std::mutex ROOT::RDF::Experimental::ProgressHelper::fSampleNameToEventEntriesMutex

mutableprivate

Definition at line 341 of file RDFHelpers.hxx.

◆ fTotalFiles

unsigned int ROOT::RDF::Experimental::ProgressHelper::fTotalFiles

private

Definition at line 348 of file RDFHelpers.hxx.

◆ fTree

std::shared_ptr<TTree> ROOT::RDF::Experimental::ProgressHelper::fTree {nullptr}

private

Definition at line 354 of file RDFHelpers.hxx.

◆ fUseShellColours

bool ROOT::RDF::Experimental::ProgressHelper::fUseShellColours

private

Definition at line 352 of file RDFHelpers.hxx.

Libraries for ROOT::RDF::Experimental::ProgressHelper:

[legend]

The documentation for this class was generated from the following files:

tree/dataframe/inc/ROOT/RDFHelpers.hxx
tree/dataframe/src/RDFHelpers.cxx

Public Member Functions

Private Member Functions

Private Attributes

Friends

Constructor & Destructor Documentation

◆ ProgressHelper()

◆ ~ProgressHelper()

Member Function Documentation

◆ ComputeCurrentFileIdx()

◆ ComputeNEventsSoFar()

◆ EvtPerSec()

◆ operator()() [1/2]

◆ operator()() [2/2]

◆ PrintProgressBar()

◆ PrintStats()

◆ PrintStatsFinal()

◆ RecordEvtCountAndTime()

◆ registerNewSample()

Friends And Related Symbol Documentation

◆ ProgressBarAction

Member Data Documentation

◆ fBarWidth

◆ fBeginTime

◆ fEventsPerSecondStatistics

◆ fEventsPerSecondStatisticsIndex

◆ fIncrement

◆ fIsTTY

◆ fLastPrintTime

◆ fLastProcessedEvents

◆ fPrintInterval

◆ fPrintMutex

◆ fProcessedEvents

◆ fSampleNameToEventEntries

◆ fSampleNameToEventEntriesMutex

◆ fTotalFiles

◆ fTree

◆ fUseShellColours