Click here to Skip to main content
65,938 articles
CodeProject is changing. Read more.
Articles / Languages / C++

A Persistent Circular Queue Buffer

2.30/5 (6 votes)
10 Aug 2022CPOL1 min read 6.1K   51  
Using the filesystem as a repository for a circular buffer for achieving persistence of data
Sample code for a persistent circular queue buffer

Introduction

The inspiration for this piece of code was the following article which talks about a persistent message broker:
https://www.oreilly.com/library/view/zeromq/9781449334437/ch04s12.html

The article had the following improvement suggestions:

  • Use a single disk file for all data, rather than multiple files. Operating systems are usually better at handling a few large files than many smaller ones.

  • Organize that disk file as a circular buffer so that new requests can be written contiguously (with very occasional wraparound). One thread, writing full speed to a disk file, can work rapidly.

Hence came up with the first step of implementing the suggestions. Currently we are able to write data in circular fashion in the disk and also read from it consistently anytime even when the program restarts.

The original code was taken from this link but it didn't implement a circular queue properly: 

https://stackoverflow.com/questions/33793535/how-to-be-sure-data-is-on-disk-after-fwrite

After modifying the codes a little I was able to get it working.

Using the code

Using the code is pretty simple.

First push some items in the queue after specifying a filename which stores the data.

C++
//  Templated declaration  
    FixedSizeQueue<int> fq;
//  Specify the filename
    fq.open("sample_file.dat", 6);

//  Push some data

    int a1[] = {0, 1, 2};
    int a2[] = {3, 4};
    fq.push_values(a1, 3);
    fq.push_values(a2, 2);

Then fetch the saved data with a get_values calls

C++
//  fetch data

int values[100];
size_t numberofvalues = fq.get_values(values);

Do remember to include the header file : circularqueue.h

Points of Interest 

Using a different file to store the head, tail and size info so that next time the program restarts, the head, tail and size can be restored properly and the queue acts as a persistent circular queue.

History

Code submitted: 11th August, 2022

Code updated: 20th August, 2022

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)