casacore
Loading...
Searching...
No Matches
Classes | Public Member Functions | Private Member Functions | Private Attributes | Static Private Attributes | List of all members
casacore::sisco::SiscoReader Class Reference

File interface for data stored in the generated model compression (Sisco) format. More...

#include <SiscoReader.h>

Classes

struct  Chunk
struct  RequestData
struct  Result
struct  BaselineData

Public Member Functions

 SiscoReader (const std::string &filename)
 ~SiscoReader ()
void Open (std::span< std::byte > header_data)
 Open the file and prepare for decompression.
size_t GetRequestBufferSize () const
void Request (size_t baseline_index, size_t n_values)
 This is an interface that allows parallelism over the decompression.
void GetNextResult (std::span< std::complex< float > > data)
 Retrieve the data associated with an earlier Request() call.
int PredictLevel () const

Private Member Functions

void Close ()
void ResultLoop ()
void ReadLoop ()
void DecompressChunk ()
void GetNextChunk (Chunk &chunk)
size_t ReadChunk (std::vector< std::byte > &buffer)
 Read current chunk and returns the decompressed size.

Private Attributes

std::set< Chunk > decompressed_queue
Chunk * current_chunk_
std::list< Chunk > chunks_
int predict_level_
size_t chunk_sequence
size_t chunk_item_position_
size_t result_counter_
size_t result_sequence_
size_t results_in_chunk_counter_
bool open_
aocommon::Lane< Chunk > read_lane_
aocommon::Lane< Chunk > decompress_lane_
aocommon::Lane< Result > result_lane_
ConditionalQueue< RequestData > request_queue_
std::set< Result > result_queue
std::thread read_thread_
std::vector< std::thread > result_workers_
std::mutex mutex_
std::set< size_t > busy_baselines_
std::map< size_t, BaselineData > baseline_data_
 Indexed by baseline_index.
std::string filename_
std::ifstream file_

Static Private Attributes

static constexpr size_t kRequestBufferSize

Detailed Description

File interface for data stored in the generated model compression (Sisco) format.

Definition at line 25 of file SiscoReader.h.

Constructor & Destructor Documentation

◆ SiscoReader()

casacore::sisco::SiscoReader::SiscoReader ( const std::string & filename)

◆ ~SiscoReader()

casacore::sisco::SiscoReader::~SiscoReader ( )

Member Function Documentation

◆ Close()

void casacore::sisco::SiscoReader::Close ( )
private

◆ DecompressChunk()

void casacore::sisco::SiscoReader::DecompressChunk ( )
private

◆ GetNextChunk()

void casacore::sisco::SiscoReader::GetNextChunk ( Chunk & chunk)
private

◆ GetNextResult()

void casacore::sisco::SiscoReader::GetNextResult ( std::span< std::complex< float > > data)

Retrieve the data associated with an earlier Request() call.

Results are returned in the same sequence as requested.

◆ GetRequestBufferSize()

size_t casacore::sisco::SiscoReader::GetRequestBufferSize ( ) const
inline

Definition at line 61 of file SiscoReader.h.

References kRequestBufferSize.

◆ Open()

void casacore::sisco::SiscoReader::Open ( std::span< std::byte > header_data)

Open the file and prepare for decompression.

This method is not thread safe and should be called (and finish) before calling GetNextResult() or Request().

◆ PredictLevel()

int casacore::sisco::SiscoReader::PredictLevel ( ) const
inline

Definition at line 84 of file SiscoReader.h.

References predict_level_.

◆ ReadChunk()

size_t casacore::sisco::SiscoReader::ReadChunk ( std::vector< std::byte > & buffer)
private

Read current chunk and returns the decompressed size.

◆ ReadLoop()

void casacore::sisco::SiscoReader::ReadLoop ( )
private

◆ Request()

void casacore::sisco::SiscoReader::Request ( size_t baseline_index,
size_t n_values )

This is an interface that allows parallelism over the decompression.

To use it, one thread should issue the requests (as fast as possible), while the main thread should obtain the values using GetNextResult(). The Request() method is blocking when the internal buffer of results is full or when all worker threads are busy.

This method is necessary because the compressed files don't store the nr of values per row or baseline_index.

Only one thread should call the Request() function, but that thread may be different than the thread calling GetNextResult().

◆ ResultLoop()

void casacore::sisco::SiscoReader::ResultLoop ( )
private

Member Data Documentation

◆ baseline_data_

std::map<size_t, BaselineData> casacore::sisco::SiscoReader::baseline_data_
private

Indexed by baseline_index.

Definition at line 137 of file SiscoReader.h.

◆ busy_baselines_

std::set<size_t> casacore::sisco::SiscoReader::busy_baselines_
private

Definition at line 129 of file SiscoReader.h.

◆ chunk_item_position_

size_t casacore::sisco::SiscoReader::chunk_item_position_
private

Definition at line 113 of file SiscoReader.h.

◆ chunk_sequence

size_t casacore::sisco::SiscoReader::chunk_sequence
private

Definition at line 112 of file SiscoReader.h.

◆ chunks_

std::list<Chunk> casacore::sisco::SiscoReader::chunks_
private

Definition at line 109 of file SiscoReader.h.

◆ current_chunk_

Chunk* casacore::sisco::SiscoReader::current_chunk_
private

Definition at line 108 of file SiscoReader.h.

◆ decompress_lane_

aocommon::Lane<Chunk> casacore::sisco::SiscoReader::decompress_lane_
private

Definition at line 121 of file SiscoReader.h.

◆ decompressed_queue

std::set<Chunk> casacore::sisco::SiscoReader::decompressed_queue
private

Definition at line 106 of file SiscoReader.h.

◆ file_

std::ifstream casacore::sisco::SiscoReader::file_
private

Definition at line 139 of file SiscoReader.h.

◆ filename_

std::string casacore::sisco::SiscoReader::filename_
private

Definition at line 138 of file SiscoReader.h.

◆ kRequestBufferSize

size_t casacore::sisco::SiscoReader::kRequestBufferSize
staticconstexprprivate

Definition at line 105 of file SiscoReader.h.

Referenced by GetRequestBufferSize().

◆ mutex_

std::mutex casacore::sisco::SiscoReader::mutex_
private

Definition at line 128 of file SiscoReader.h.

◆ open_

bool casacore::sisco::SiscoReader::open_
private

Definition at line 118 of file SiscoReader.h.

◆ predict_level_

int casacore::sisco::SiscoReader::predict_level_
private

Definition at line 111 of file SiscoReader.h.

Referenced by PredictLevel().

◆ read_lane_

aocommon::Lane<Chunk> casacore::sisco::SiscoReader::read_lane_
private

Definition at line 120 of file SiscoReader.h.

◆ read_thread_

std::thread casacore::sisco::SiscoReader::read_thread_
private

Definition at line 126 of file SiscoReader.h.

◆ request_queue_

ConditionalQueue<RequestData> casacore::sisco::SiscoReader::request_queue_
private

Definition at line 123 of file SiscoReader.h.

◆ result_counter_

size_t casacore::sisco::SiscoReader::result_counter_
private

Definition at line 114 of file SiscoReader.h.

◆ result_lane_

aocommon::Lane<Result> casacore::sisco::SiscoReader::result_lane_
private

Definition at line 122 of file SiscoReader.h.

◆ result_queue

std::set<Result> casacore::sisco::SiscoReader::result_queue
private

Definition at line 124 of file SiscoReader.h.

◆ result_sequence_

size_t casacore::sisco::SiscoReader::result_sequence_
private

Definition at line 115 of file SiscoReader.h.

◆ result_workers_

std::vector<std::thread> casacore::sisco::SiscoReader::result_workers_
private

Definition at line 127 of file SiscoReader.h.

◆ results_in_chunk_counter_

size_t casacore::sisco::SiscoReader::results_in_chunk_counter_
private

Definition at line 116 of file SiscoReader.h.

The documentation for this class was generated from the following file:
Generated by doxygen 1.15.0