Solved aio_read() causes system crash

Hello,
I newbie in FreeBSD programming. (and using heh) So have sometimes a "stupid" problems.
I try to change my project from blocked file access to asyncronous. I add two new (aio) functions similar to blocked ones to my file access class. And add aio_load="YES" to /boot/loader.conf. But when iI use "aio" functions, it causes system crash on some (10..15-th) aio_read() call.

So there is my file access class:
Code:
class FileDB {
public:
...
    void read( int key ); // blocked read
    void read_aio( int key ); // async read
    void getLastKeyData(uint32_t* buffer); //getting data after blocked read
    void getLastKeyData_aio(uint32_t* buffer); // getting data after async read
    int getLastKeyRecordCount(){ return count; }; // last reading operation records count
private:
    int fd; // file des
    bool volatile asyncOpInProgress; // flag of async op
    int sectorSize;   // i use 4K blocks in file.
    uint64_t currentIdxSector;
    uint64_t currentDataSector;
    uint32_t* tmpIdx;
    uint64_t* dataSectorBuf;
    uint64_t lastDataPos;
    int rdOffs;
    int rdSize;
    std::string aFileName;
    int count; // кол-во записей, указанных в индексе последнего чтения.
    struct aiocb aio;
    struct aiocb *cblist[1];
    int err_r;
}
Both blocked and async functions are similar but this piece of code:
Code:
void FileDB::read( int key )
{   
..... bla bla bla....
    if (currentDataSector != sectorNum)
    { // if that "sector" of data wasn't read allready, then read it
        lseek( fd, sectorNum * sectorSize, SEEK_SET );
        rdSize_l = (rdSize_l > sectorSize) ? sectorSize * 2 : sectorSize;
        ::read( fd, dataSectorBuf, rdSize_l  );
        currentDataSector = sectorNum;
    }
}

void FileDB::read_aio( int key )
{
..... bla bla bla....
    memset(&aio, 0, sizeof( aio ));     // zeroing the aiocb
    if (currentDataSector != sectorNum)
    {// if that "sector" of data wasn't read allready, then read it
        aio.aio_fildes = fd;
        aio.aio_offset = sectorSize * sectorNum;   
        aio.aio_buf = (char*)tmpIdx;
        rdSize_l = (rdSize_l > sectorSize) ? sectorSize * 2 : sectorSize;
        aio.aio_nbytes = rdSize_l;
        aio_read(&aio);
        currentDataSector = sectorNum;
        asyncOpInProgress = true;// setting the async flag to indicate to getData func that something was readed
    }
}

// Getting the data
void FileDB::getLastKeyData(uint32_t* buffer)
{
    memcpy( buffer, ((char*)dataSectorBuf)+rdOffs, rdSize );
    .... bla bla bla ....
}

void FileDB::getLastKeyData_aio(uint32_t* buffer)
{
    if( asyncOpInProgress ) // if there was real data reading, then awaiting for operation completion
    {
        while( (err_r = aio_error(&aio)) == EINPROGRESS )
        {
            cblist[0] = &aio;
            aio_suspend( cblist, 1, NULL );
        }
        if (err_r == 0)
        {
            aio_return(&aio);                   
        }
//      for the first time (till the aio will work fine) we ignore  the fact that something may not
//      reads. (err_r !=0). dataSectorBuf are constantly present forr all object life cycle,
//      so in a bad case we getting the garbage to buffer. ;)
        asyncOpInProgress = false; // clearing the sync flag (raeding operation was completed)
    }
    memcpy( buffer, ((char*)dataSectorBuf)+rdOffs, rdSize );
       .... bla bla bla ....
}

So when i use it like that, all works fine:
Code:
        FileDB db = new FileDB(); //
        uint32_t* tmpbuf = uint32_t[2048]; // 2k*4 = 2 "sectors"
        ...
        for(...){
        ....
            db->read(currIdx);
            db->getLastKeyData(tmpbuf);
            recCnt = db->getLastKeyRecordCount();       
        .....
        }
        delete db;
        delete []tmpbuf;

But when iI use it like that, it crash the system at 10-15 cycle step.
Code:
          FileDB db = new FileDB(); //
        uint32_t* tmpbuf = uint32_t[2048]; // 2k*4 = 2 "sectors"
        ...
        for(...){
            db->read_aio(currIdx);
            db->getLastKeyData_aio(tmpbuf);
            recCnt = db->getLastKeyRecordCount();       
         ....
        }
        delete db;
        delete []tmpbuf;
Moreover iI follow all steps in debugger, so it hangs really on "aio_read(&aio);" line. Not the first or second time, but 10-15th.

Thus, as iI see. iI have made something wrong in code or made (didn't made) something in system configuration. So can anybody point me to my fail?

P.S. I use in the code term "sector" cause this is not the first project revision. The first one uses RAW partition access. By blocks of 4K. In this project all data placed in files on normal UFS disk, but we still use it by 4K blocks and call it "sectors" ;)
 
it causes system crash
I probably misunderstand you, but what do you mean by system crash? Does FreeBSD panic? If so open a bug report on https://bugs.freebsd.org/. AFAIK using aio_read(2) should not cause the kernel to panic even if used incorrectly.

Code:
aio.aio_buf = (char*)tmpIdx;
rdSize_l = (rdSize_l > sectorSize) ? sectorSize * 2 : sectorSize;
aio.aio_nbytes = rdSize_l;
Is tmpIdx large enough? Why are you not using dataSectorBuf in read_aio as you did in read?
 
Is tmpIdx large enough? Why are you not using dataSectorBuf in read_aio as you did in read?
I'm sorry iI just don't want to place lage code. Both read functions actually make up to two reads. The firs from "index area" (to tmpIdx) and the next from data area (to dataSectorBuf)
Both reading code is similar (but io operation waiting cycle for index area is placed in read_aio(), when the waiting cycle for data area placed separated in getLastKeyData_aio() ) So i change it in read() function, but forget to change in read_aio() before place it to forum...:oops:

So you right there must be:
Code:
void FileDB::read_aio( int key )
{
     ..... bla bla bla....
     memset(&aio, 0, sizeof( aio )); // zeroing the aiocb
     if (currentDataSector != sectorNum)
     {// if that "sector" of data wasn't read allready, then read it
           aio.aio_fildes = fd;
           aio.aio_offset = sectorSize * sectorNum;
           aio.aio_buf = (char*)dataSectorBuf;
           rdSize_l = (rdSize_l > sectorSize) ? sectorSize * 2 : sectorSize;
           aio.aio_nbytes = rdSize_l;
           aio_read(&aio);
           currentDataSector = sectorNum;
           asyncOpInProgress = true;// setting the async flag to indicate to getData func that something was readed
     }
}

the both dataSectorBuf and tmpIdx are allocated large enough ( ... = new uint32_t[2048]; //= 8K each );

about "system panic". I don't sure...but i think that is it. The system shows counter and said that all data will be dumped then computer restarts...
 
I'm sorry iI did not find how to mark topic as "Solved"?

P.S. Thank you tobik, it was really a mistake in tmpIdx with place dataSectorBuf (that is !@#$$ copy/paste ...:oops: )
 
Last edited by a moderator:
Back
Top