XRootD
Loading...
Searching...
No Matches
XrdPfc::IOFile Class Reference

Downloads original file into a single file on local disk. Handles read requests as they come along. More...

#include <XrdPfcIOFile.hh>

+ Inheritance diagram for XrdPfc::IOFile:
+ Collaboration diagram for XrdPfc::IOFile:

Public Member Functions

 IOFile (XrdOucCacheIO *io, Cache &cache)
 
 ~IOFile ()
 
void DetachFinalize () override
 Abstract virtual method of XrdPfc::IO Called to destruct the IO object after it is no longer used.
 
long long FSize () override
 
int Fstat (struct stat &sbuff) override
 
bool HasFile () const
 Check if File was opened successfully.
 
bool ioActive () override
 Abstract virtual method of XrdPfc::IO Called to check if destruction needs to be done in a separate task.
 
virtual int pgRead (char *buff, long long offs, int rdlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
 
void pgRead (XrdOucCacheIOCB &iocb, char *buff, long long off, int size, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0) override
 
int Read (char *buff, long long off, int size) override
 Pass Read request to the corresponding File object.
 
void Read (XrdOucCacheIOCB &iocb, char *buff, long long off, int size) override
 
int ReadV (const XrdOucIOVec *readV, int n) override
 Pass ReadV request to the corresponding File object.
 
void ReadV (XrdOucCacheIOCB &iocb, const XrdOucIOVec *readV, int n) override
 
void Update (XrdOucCacheIO &iocp) override
 
- Public Member Functions inherited from XrdPfc::IO
 IO (XrdOucCacheIO *io, Cache &cache)
 
virtual XrdOucCacheIOBase ()
 Original data source.
 
bool Detach (XrdOucCacheIOCD &iocdP) final
 
XrdOucCacheIOGetInput ()
 
const char * GetLocation ()
 
XrdSysTraceGetTrace ()
 
const char * Path () override
 Original data source URL.
 
int Sync () override
 
virtual void Sync (XrdOucCacheIOCB &iocb)
 
int Trunc (long long Offset) override
 
virtual void Trunc (XrdOucCacheIOCB &iocb, long long offs)
 
int Write (char *Buffer, long long Offset, int Length) override
 
virtual void Write (XrdOucCacheIOCB &iocb, char *buff, long long offs, int wlen)
 
- Public Member Functions inherited from XrdOucCacheIO
 XrdOucCacheIO ()
 Construct and Destructor.
 
virtual const char * Location (bool refresh=false)
 
virtual int pgWrite (char *buff, long long offs, int wrlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
 
virtual void pgWrite (XrdOucCacheIOCB &iocb, char *buff, long long offs, int wrlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
 
virtual void Preread (aprParms &Parms)
 
virtual void Preread (long long offs, int rlen, int opts=0)
 
virtual int WriteV (const XrdOucIOVec *writV, int wnum)
 
virtual void WriteV (XrdOucCacheIOCB &iocb, const XrdOucIOVec *writV, int wnum)
 

Additional Inherited Members

- Static Public Attributes inherited from XrdOucCacheIO
static const uint64_t forceCS = 0x0000000000000001ULL
 
static const int SingleUse = 0x0001
 Mark pages for single use.
 
- Protected Member Functions inherited from XrdPfc::IO
std::string GetFilename ()
 
const char * GetPath ()
 
unsigned short ObtainReadSid ()
 
const char * RefreshLocation ()
 
bool register_block_error (int res)
 
bool register_incomplete_read ()
 
- Protected Member Functions inherited from XrdOucCacheIO
virtual ~XrdOucCacheIO ()
 
- Protected Attributes inherited from XrdPfc::IO
RAtomic_int m_active_read_reqs
 number of active read requests
 
Cachem_cache
 reference to Cache object
 
std::map< int, int > m_error_counts
 
int m_incomplete_count {0}
 
const char * m_traceID
 

Detailed Description

Downloads original file into a single file on local disk. Handles read requests as they come along.

Definition at line 39 of file XrdPfcIOFile.hh.

Constructor & Destructor Documentation

◆ IOFile()

IOFile::IOFile ( XrdOucCacheIO * io,
Cache & cache )

Definition at line 36 of file XrdPfcIOFile.cc.

36 :
37 IO(io, cache),
38 m_file(0)
39{
40 m_file = Cache::GetInstance().GetFile(GetFilename(), this);
41}
File * GetFile(const std::string &, IO *, long long off=0, long long filesize=0)
Definition XrdPfc.cc:415
static Cache & GetInstance()
Singleton access.
Definition XrdPfc.cc:163
std::string GetFilename()
Definition XrdPfcIO.hh:56
IO(XrdOucCacheIO *io, Cache &cache)
Definition XrdPfcIO.cc:6

References XrdPfc::IO::IO(), XrdOucCacheIO::XrdOucCacheIO(), XrdPfc::Cache::GetFile(), XrdPfc::IO::GetFilename(), and XrdPfc::Cache::GetInstance().

Referenced by pgRead(), Read(), and ReadV().

+ Here is the call graph for this function:
+ Here is the caller graph for this function:

◆ ~IOFile()

IOFile::~IOFile ( )

Definition at line 44 of file XrdPfcIOFile.cc.

45{
46 // called from Detach() if no sync is needed or
47 // from Cache's sync thread
48 TRACEIO(Debug, "~IOFile() " << this);
49}
#define TRACEIO(act, x)
bool Debug

References Debug, and TRACEIO.

Member Function Documentation

◆ DetachFinalize()

void IOFile::DetachFinalize ( )
overridevirtual

Abstract virtual method of XrdPfc::IO Called to destruct the IO object after it is no longer used.

Implements XrdPfc::IO.

Definition at line 114 of file XrdPfcIOFile.cc.

115{
116 // Effectively a destructor.
117
118 TRACE(Debug, "DetachFinalize() " << this);
119
120 m_file->RequestSyncOfDetachStats();
121 Cache::GetInstance().ReleaseFile(m_file, this);
122
123 if (( ! m_error_counts.empty() || m_incomplete_count > 0) && XRD_TRACE What >= TRACE_Error) {
124 char info[1024];
125 int pos = 0, cap = 1024;
126 bool truncated = false;
127 for (auto [err, cnt] : m_error_counts) {
128 int len = snprintf(&info[pos], cap, " ( %d : %d)", err, cnt);
129 if (len >= cap) {
130 truncated = true;
131 break;
132 }
133 pos += len;
134 cap -= len;
135 }
136 TRACE(Error, "DetachFinalize() " << this << " n_incomplete_reads=" << m_incomplete_count
137 << ", block (error : count) report:" << info << (truncated ? " - truncated" : ""));
138 }
139
140 delete this;
141}
#define TRACE_Error
Definition XrdPfcTrace.hh:7
#define XRD_TRACE
#define TRACE(act, x)
Definition XrdTrace.hh:63
void ReleaseFile(File *, IO *)
Definition XrdPfc.cc:497
int m_incomplete_count
Definition XrdPfcIO.hh:90
std::map< int, int > m_error_counts
Definition XrdPfcIO.hh:91

References Debug, Error, XrdPfc::Cache::GetInstance(), XrdPfc::IO::m_error_counts, XrdPfc::IO::m_incomplete_count, XrdPfc::Cache::ReleaseFile(), TRACE, TRACE_Error, and XRD_TRACE.

+ Here is the call graph for this function:

◆ FSize()

long long IOFile::FSize ( )
overridevirtual

Obtain size of the file.

Returns
Size of the file in bytes.

Implements XrdOucCacheIO.

Definition at line 66 of file XrdPfcIOFile.cc.

67{
68 return m_file->GetFileSize();
69}

◆ Fstat()

int IOFile::Fstat ( struct stat & sbuff)
overridevirtual

Perform an fstat() operation (defaults to passthrough).

Parameters
sbuffreference to the stat buffer to be filled in. Only fields st_size, st_blocks, st_mtime (st_atime and st_ctime may be set to st_mtime), st_ino, and st_mode need to be set. All other fields are preset and should not be changed.
Returns
<0 - fstat failed, value is -errno. =0 - fstat succeeded, sbuff holds stat information. >0 - fstat could not be done, forward operation to next level.

Reimplemented from XrdOucCacheIO.

Definition at line 52 of file XrdPfcIOFile.cc.

53{
54 // This is only called during Cache::Attach / Cache::GetFile() for file creation.
55 // Should really be separate name but one needs to change virtual interface
56 // so initialStat() becomes virtual in the base-class.
57 // Also, IOFileBlock should be ditched.
58 if ( ! m_file) {
59 return initialStat(sbuff);
60 }
61
62 return m_file->Fstat(sbuff);
63}

References stat.

◆ HasFile()

bool XrdPfc::IOFile::HasFile ( ) const
inline

Check if File was opened successfully.

Definition at line 49 of file XrdPfcIOFile.hh.

49{ return m_file != 0; }

Referenced by XrdPfc::Cache::Attach().

+ Here is the caller graph for this function:

◆ ioActive()

bool IOFile::ioActive ( )
overridevirtual

Abstract virtual method of XrdPfc::IO Called to check if destruction needs to be done in a separate task.

Implements XrdPfc::IO.

Definition at line 107 of file XrdPfcIOFile.cc.

108{
110 return m_file->ioActive(this);
111}
const char * RefreshLocation()
Definition XrdPfcIO.hh:57

References XrdPfc::IO::RefreshLocation().

+ Here is the call graph for this function:

◆ pgRead() [1/2]

int XrdOucCacheIO::pgRead ( char * buff,
long long offs,
int rdlen,
std::vector< uint32_t > & csvec,
uint64_t opts = 0,
int * csfix = 0 )
virtual

Reimplemented from XrdOucCacheIO.

Definition at line 190 of file XrdOucCache.cc.

45{
46 (void)csfix;
47 int bytes;
48
49// Read the data into the buffer
50//
51 bytes = Read(buff, offs, rdlen);
52
53// Calculate checksums if so wanted
54//
55 if (bytes > 0 && (opts & forceCS))
56 XrdOucPgrwUtils::csCalc((const char *)buff, (ssize_t)offs,
57 (size_t)bytes, csvec);
58
59// All done
60//
61 return bytes;
62}
struct myOpts opts
static const uint64_t forceCS
static void csCalc(const char *data, off_t offs, size_t count, uint32_t *csval)
int Read(char *buff, long long off, int size) override
Pass Read request to the corresponding File object.

◆ pgRead() [2/2]

void IOFile::pgRead ( XrdOucCacheIOCB & iocb,
char * buff,
long long offs,
int rdlen,
std::vector< uint32_t > & csvec,
uint64_t opts = 0,
int * csfix = 0 )
overridevirtual

Read file pages and checksums using asynchronous I/O (default sync).

Parameters
iocbreference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
buffpointer to buffer where the bytes are to be placed.
offsThe offset where the read is to start.
rdlenThe number of bytes to read.
csvecA vector which will be filled with the corresponding CRC32C checksum for each page or page segment.
optsProcessing options: forceCS - always return checksums even when not available.
csfixWhen not nil, returns the number of corrected checksum errs.

Reimplemented from XrdOucCacheIO.

Definition at line 196 of file XrdPfcIOFile.cc.

198{
199 struct ZHandler : ReadReqRH
200 { using ReadReqRH::ReadReqRH;
201 IOFile *m_io = nullptr;
202 std::function<void (int)> m_lambda {0};
203
204 void Done(int result) override {
205 if (m_lambda) m_lambda(result);
206 m_io->ReadEnd(result, this);
207 }
208 };
209
211
212 auto *rh = new ZHandler(ObtainReadSid(), &iocb);
213 rh->m_io = this;
214
215 TRACEIO(Dump, "pgRead() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
216
218 rh->m_lambda = [=, &csvec](int result) {
219 if (result > 0)
220 XrdOucPgrwUtils::csCalc((const char *)buff, (ssize_t)off, (size_t)result, csvec);
221 };
222
223 int retval = ReadBegin(buff, off, size, rh);
224 if (retval != -EWOULDBLOCK)
225 {
226 rh->Done(retval);
227 }
228}
virtual void Done(int result)=0
RAtomic_int m_active_read_reqs
number of active read requests
Definition XrdPfcIO.hh:72
unsigned short ObtainReadSid()
Definition XrdPfcIO.hh:59
ReadReqRH(unsigned short sid, XrdOucCacheIOCB *iocb)
Definition XrdPfcFile.hh:67

References IOFile(), XrdPfc::ReadReqRH::ReadReqRH(), XrdOucPgrwUtils::csCalc(), XrdOucCacheIO::forceCS, Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), opts, and TRACEIO.

+ Here is the call graph for this function:

◆ Read() [1/2]

int IOFile::Read ( char * buff,
long long off,
int size )
overridevirtual

Pass Read request to the corresponding File object.

Implements XrdOucCacheIO.

Definition at line 149 of file XrdPfcIOFile.cc.

150{
152
153 auto *rh = new ReadReqRHCond(ObtainReadSid(), nullptr);
154
155 TRACEIO(Dump, "Read() sync " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
156
157 rh->m_cond.Lock();
158 int retval = ReadBegin(buff, off, size, rh);
159 if (retval == -EWOULDBLOCK)
160 {
161 rh->m_cond.Wait();
162 retval = rh->m_retval;
163 }
164 rh->m_cond.UnLock();
165
166 return ReadEnd(retval, rh);
167}

References Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

+ Here is the call graph for this function:

◆ Read() [2/2]

void IOFile::Read ( XrdOucCacheIOCB & iocb,
char * buff,
long long offs,
int rlen )
overridevirtual

Perform an asynchronous read (defaults to synchronous).

Parameters
iocbreference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
buffpointer to the buffer to receive the results. The buffer must remain valid until the callback is invoked.
offsthe offset into the file.
rlenthe number of bytes to read.

Reimplemented from XrdOucCacheIO.

Definition at line 170 of file XrdPfcIOFile.cc.

171{
172 struct ZHandler : ReadReqRH
173 { using ReadReqRH::ReadReqRH;
174 IOFile *m_io = nullptr;
175
176 void Done(int result) override {
177 m_io->ReadEnd(result, this);
178 }
179 };
180
182
183 auto *rh = new ZHandler(ObtainReadSid(), &iocb);
184 rh->m_io = this;
185
186 TRACEIO(Dump, "Read() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
187
188 int retval = ReadBegin(buff, off, size, rh);
189 if (retval != -EWOULDBLOCK)
190 {
191 rh->Done(retval);
192 }
193}

References IOFile(), XrdPfc::ReadReqRH::ReadReqRH(), Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

+ Here is the call graph for this function:

◆ ReadV() [1/2]

int IOFile::ReadV ( const XrdOucIOVec * readV,
int n )
overridevirtual

Pass ReadV request to the corresponding File object.

Reimplemented from XrdOucCacheIO.

Definition at line 275 of file XrdPfcIOFile.cc.

276{
278
279 auto *rh = new ReadReqRHCond(ObtainReadSid(), nullptr);
280
281 TRACEIO(Dump, "ReadV() sync " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " n_chunks: " << n);
282
283 rh->m_cond.Lock();
284 int retval = ReadVBegin(readV, n, rh);
285 if (retval == -EWOULDBLOCK)
286 {
287 rh->m_cond.Wait();
288 retval = rh->m_retval;
289 }
290 rh->m_cond.UnLock();
291 return ReadVEnd(retval, rh);
292}

References Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::ReadReqRH::m_seq_id, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

+ Here is the call graph for this function:

◆ ReadV() [2/2]

void IOFile::ReadV ( XrdOucCacheIOCB & iocb,
const XrdOucIOVec * readV,
int rnum )
overridevirtual

Perform an asynchronous vector read (defaults to synchronous).

Parameters
iocbreference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
readVpointer to a vector of read requests.
rnumthe number of elements in the vector.

Reimplemented from XrdOucCacheIO.

Definition at line 295 of file XrdPfcIOFile.cc.

296{
297 struct ZHandler : ReadReqRH
298 { using ReadReqRH::ReadReqRH;
299 IOFile *m_io = nullptr;
300
301 void Done(int result) override { m_io-> ReadVEnd(result, this); }
302 };
303
305
306 auto *rh = new ZHandler(ObtainReadSid(), &iocb);
307 rh->m_io = this;
308
309 TRACEIO(Dump, "ReadV() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " n_chunks: " << n);
310
311 int retval = ReadVBegin(readV, n, rh);
312 if (retval != -EWOULDBLOCK)
313 {
314 rh->Done(retval);
315 }
316}

References IOFile(), XrdPfc::ReadReqRH::ReadReqRH(), XrdOucCacheIOCB::Done(), Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::ReadReqRH::m_seq_id, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

+ Here is the call graph for this function:

◆ Update()

void IOFile::Update ( XrdOucCacheIO & iocp)
overridevirtual

Update the originally passed XrdOucCacheIO object with the object passed. All future uses underlying XrdOucCacheIO object must now use this object. Update() is called when Prepare() indicated that the file should not be physically opened and a file method was invoked in the XrdOucCacheIO passed to Attach(). When this occurs, the file is actually opened and Update() called to replace the original XrdOucCacheIO object with one that uses the newly opened file.

Parameters
iocpreference to the new XrdOucCacheIO object.

Reimplemented from XrdPfc::IO.

Definition at line 100 of file XrdPfcIOFile.cc.

101{
102 IO::Update(iocp);
103 m_file->ioUpdated(this);
104}
void Update(XrdOucCacheIO &iocp) override
Definition XrdPfcIO.cc:16

References XrdOucCacheIO::XrdOucCacheIO(), and XrdPfc::IO::Update().

+ Here is the call graph for this function:

The documentation for this class was generated from the following files: