securelocker: an httplite pretty secure file storage locker

• Download source - 58.8 KB

Introduction

This project shows how you can weave httplite REST APIs with off-the-shelf security components to build a "pretty secure" file storage application. The application may well have no use in the computer and networking world it lives in but it was fun to write and get to work, and some interesting code fell out, so enjoy.

Shoulders We're Standing On

• Cross-platform GUID generation
• SHA-256 hashing
• Blowfish encryption

securelocker - Pretty Secure File Storage Locker

The idea with securelocker is that we'd like to model a software system after the real world businesses that rent lockers for people to store and exchange items. You pay the fee, you get a locker number and a key to the locker. You put your stuff in it, lock it up, then maybe you give the number and key to somebody else as an exchange, or maybe you just come back later to retrieve your stuff.

Important: The locker company keeps a copy of the key you got. So it's pretty secure: it's as secure as you trust the locker company.

To model this in C++, I decided to write two applications. System operators run securelockerd to run the locker business, and end users operate securelocker to work with their lockers. A lot happens out-of-band of the computer commands and network operations.

securelockerd

securelockerd is the server-side command line application that you use to run the locker business. The application takes commands to register, checkin, and checkout clients. It also runs an httplite web server for handling clients running securelocker, the client-side program described below.

The register command is the phone call before you go to the locker business, reserving a locker. This command takes the name of the person making the reservation, and stores it in the leger described below. The client would convey their name outside of the application, out-of-band. I'm not sure of the value of this step, it might be unnecessary. Chime in with your thoughts on that.

The checkin command finds a free locker, creates the key to the locker, and stores the room number and key to the locker in the leger and gives the number and key to the operator to give to the client out-of-band. You may gather that this system employs shared secret symmetric encryption, Blowfish.

The checkout command empties the locker and erases the record from the leger.

There is a leger object where locker information is managed. This leger is shared between the command processing code and the web server. The leger is thread-safe. The leger data is stored on disk in an encrypted file. When the operator launches securelockerd, it prompts the user for the password to the leger file, and loads the leger into memory. All changes made to the leger write the leger file back out. This is obviously not designed for thousands of lockers, but it should be adequate for hundreds of lockers, sort of in line with a real locker business.

class locker leger
{
public: // interface
	lockerleger(const std::wstring& password, const std::wstring& legerFilePath);
	void load();

	void registerClient(const std::wstring& name);

	void checkin(const std::wstring& name, uint32_t& room, std::string& key);

	void checkout(const std::wstring& name, uint32_t& room);

	struct legerentry
	{
		legerentry();
		legerentry(const std::wstring& _name, uint32_t _room, const std::string& _key);
		legerentry(const std::wstring& str);

		std::wstring toString() const;

		std::wstring name;
		uint32_t room;
		std::string key;
	};

private: // implementation
	void save();

	bool isNameRegistered(const std::wstring& name) const;
	
	std::shared_ptr<legerentry> getNameEntry(const std::wstring& name) const;
	
	uint32_t getAvailableRoom() const;

private: // state
	std::vector<std::shared_ptr<legerentry>> m_entries;

	mutable std::mutex m_mutex;

	const std::string m_password;
	const std::wstring m_legerFilePath;
};

The algorithm for finding an available room is fun, a decent interview question:

uint32_t lockerleger::getAvailableRoom() const
{
	// Get the sorted list of all rooms
	std::vector<uint32_t> rooms;
	for (const auto& legerentry : m_entries)
	{
		if (legerentry->room > 0)
			rooms.push_back(legerentry->room);
	}
	std::sort(rooms.begin(), rooms.end());

	// Walk the list looking for a gap in the sequence
	uint32_t last = 0;
	for (uint32_t cur : rooms)
	{
		uint32_t should = last + 1;
		if (cur > should) // past empty spot
			return should;
		last = cur;
	}

	// Failing that, go one after the end
	return last + 1;
}

Locker contents are stored in a directory, one subdirectory per locker. Each locker contains a flat list of files, no subdirectories. This made implementation simpler. If you want subdirectories, use a ZIP file. If you want better security, use a password-protected ZIP file. Even with a password-protected ZIP file, if you want to exchange the file, you're going to need to share the password with the other party in some way. It's all in who you trust.

securelockerd needs to get the leger password from the user at program startup. As the leger contents are highly sensitive, password entry should not display the user's password on the screen. To accomplish this, you #include <conio.h> and use a blob of code like this:

printf("Leger Access Password: ");
std::wstring password;
while (true)
{
	char c = _getch();
	if (c == '\r' || c == '\n')
		break;
	password += c;
	printf("*");
}
printf("\n");

Before receiving commands from the user, the program loads the leger, and starts the web server:

std::wstring lockerRootDir = argv[2];
std::wstring legerFilePath = fs::path(lockerRootDir).append("leger.dat");
lockerleger leger(password, legerFilePath);
leger.load();

lockerserver server
(
	static_cast<uint16_t>(port), 
	leger, 
	lockerRootDir
);
server.start();

The code for preventing malicious file access is pretty cute:

bool securelib::IsFilenameValid(const std::wstring& filename)
{
	if (filename.empty())
		return false;

	if
	(
		filename == L"."
		||
		filename.front() == ' '
		||
		filename.back() == ' '
		||
		filename.back() == '.'
		||
		filename.find(L"..") != std::wstring::npos
	)
	{
		return false;
	}

	for (auto c : filename)
	{
		if ((c >= 0x0 && c <= 0x1F) || c == 0x7F)
			return false;

		switch (static_cast<char>(c))
		{
		case '\"':
		case '\\':
		case ':':
		case '/':
		case '<':
		case '>':
		case '|':
		case '?':
		case '*':
			return false;
		}
	}
	return true;
}

securelocker

securelocker is the client-side command line program where you can:

1. put
2. dir
3. get
4. del

files in the locker.

I implemented a basic httplite wrapper class to implement these functions so that the app just does command line stuff:

void lockerclient::put(const std::wstring& filename, const std::vector<uint8_t>& bytes)
{
	Request request;
	request.Verb = "PUT";
	request.Path.push_back(filename);
	request.Payload.emplace(Encrypt(bytes, m_key));
	doHttp(request);
}
	
std::vector<std::wstring> lockerclient::dir()
{
	Request request;
	request.Verb = "DIR"; // nonstandard, works well for us
	request.Path.push_back(L"/");
	Response response = doHttp(request);
	std::wstring dirResult =
		response.Payload.has_value() ? response.Payload->ToString() : L"";
	return Split(dirResult, L'\n');
}

std::vector<uint8_t> lockerclient::get(const std::wstring& filename)
{
	Request request;
	request.Verb = "GET";
	request.Path.push_back(filename);
	Response response = doHttp(request);
	return Decrypt(response.Payload->Bytes, m_key);
}

void lockerclient::del(const std::wstring& filename)
{
	Request request;
	request.Verb = "DELETE";
	request.Path.push_back(filename);
	doHttp(request);
}

Response lockerclient::doHttp(Request& request)
{
	Response response =
		issueClientHttpCommand
		(
			m_client,
			m_room,
			m_key,
			request
		);
	if (response.GetStatusCode() / 100 != 2)
		throw std::runtime_error(("HTTP operation failed: " + response.Status).c_str());
	return response;
}

...but what pray tell is issueClientHttpCommand? It implements the client side of a challenge-response authentication mechanism:

static Response issueClientHttpCommand
(
	httplite::HttpClient& client,
	uint32_t room,
	const std::string& key,
	httplite::Request& request
)
{
	trace(L"Client HTTP Command: " + std::to_wstring(room) + L" - " + 
          toWideStr(request.Verb) + L" - " + request.Path[0]);
	request.Headers["X-Room-Number"] = std::to_string(room); // we send every time, 
                                       // but the server only believes us once

	bool gotChallenge = false;
	bool submittedChallenge = false;

	std::string challengePhrase;
	std::string challengeNonce;

	while (true) // process authentication challenges then the actual request
	{
		if (gotChallenge)
		{
			auto encryptedResponse =
				Encrypt
				(
					StrToVec
					(
						std::to_string(room) +
						challengePhrase +
						challengeNonce
					),
					key
				);
			std::string challengeResponse =
				Hash(encryptedResponse.data(), encryptedResponse.size());
			request.Headers["X-Challenge-Response"] = challengeResponse;
			submittedChallenge = true;
			trace("Got challenge, response: " + challengeResponse);
		}

		trace("Issuing request...");
		Response response = client.ProcessRequest(request);
		trace("Response: " + response.Status);
		uint16_t statusCode = response.GetStatusCode();
		if (statusCode / 100 == 2)
		{
			// Authentication is fine, so remove the auth headers 
			// to keep subsequent requests clean
			request.Headers.erase("X-Room-Number");
			request.Headers.erase("X-Challenge-Response");
			return response;
		}
		else if (statusCode / 100 == 4)
		{
			// no double-dipping, you get one shot
			// client expects a successful response, 
			// so we throw instead for return response
			if (submittedChallenge)
				throw std::runtime_error("Access denied.");

			gotChallenge = true;
			challengePhrase = response.Headers["X-Challenge-Phrase"];
			challengeNonce = response.Headers["X-Challenge-Nonce"];
		}
		else
			throw std::runtime_error(("Unregonized Server Response: " + 
                                       response.Status).c_str());
	}
}

The server side of authentication is sort of a mirror image. I added ConnectionVariables to httplite to get slipped from the per-socket-thread's local variable into the Request object. It's sort of session variables without cookies, just state persisted on each socket thread in the server. Not scalable, but very simple.

static std::shared_ptr<Response> authServerHttpRequest 
(
	const Request& request,
	std::function<int()> nonceGen,
	std::function<std::string(uint32_t room)> keyGet
)
{
	// Make connection variables easier to work with
	auto& connVars = *request.ConnectionVariables;

	// Bail if the client is already authenticated
	if (connVars.find(L"Authenticated") != connVars.end())
	{
		trace("Auth: Client authenticated");
		return nullptr;
	}

	// Unpack the challenge connection vars
	auto roomIt = connVars.find(L"RoomNumber");
	auto challengeIt = connVars.find(L"ChallengePhrase");
	auto nonceIt = connVars.find(L"ChallengeNonce");
	if
	(
		roomIt == connVars.end()
		||
		challengeIt == connVars.end()
		||
		nonceIt == connVars.end()
	)
	{
		trace("Auth: Client not challenged yet");

		// Get the room number from the request and validate it
		// NOTE: This is the only time when the room number is read from the client
		//		 We can't allow clients to change which room they're talking about 
		//		 later and gain access to another room's contents, password or not
		auto roomRequestIt = request.Headers.find("X-Room-Number");
		if (roomRequestIt == request.Headers.end())
		{
			trace("Auth: No room number");
			std::shared_ptr<Response> response = std::make_shared<Response>();
			response->Status = "403 Invalid Request";
			return response;
		}
		int roomInt = atoi(roomRequestIt->second.c_str());
		if (roomInt <= 0)
		{
			trace("Auth: Invalid room number");
			std::shared_ptr<Response> response = std::make_shared<Response>();
			response->Status = "403 Invalid Request";
			return response;
		}

		// Create the challenge
		std::string challenge = UniqueStr();
		std::string nonce = std::to_string(nonceGen());
		trace("Auth: Challenge: " + challenge + " - " + nonce);

		// Stash the challenge in connections vars
		connVars[L"RoomNumber"] = std::to_wstring(roomInt);
		connVars[L"ChallengePhrase"] = toWideStr(challenge);
		connVars[L"ChallengeNonce"] = toWideStr(nonce);

		// Return the challenge response
		std::shared_ptr<Response> response = std::make_shared<Response>();
		response->Status = "401 Access Denied";
		response->Headers["X-Challenge-Phrase"] = challenge;
		response->Headers["X-Challenge-Nonce"] = nonce;
		return response;
	}
	else // we have a complete set of challenge connections vars
	{
		trace("Auth: Client challenged");

		// Unpack the server challenge from the connection vars
		uint32_t room = static_cast<uint32_t>(_wtoi(roomIt->second.c_str()));
		std::string challengePhrase = toNarrowStr(challengeIt->second);
		std::string challengeNonce = toNarrowStr(nonceIt->second);

		// Erase the challenge connection vars to prevent clients from hammering
		// on the same challenge until they find something that works
		connVars.erase(roomIt);
		connVars.erase(challengeIt);
		connVars.erase(nonceIt);

		// Get the client's response to the server challenge
		std::string challengeClientResponse;
		{
			auto requestChallengeResponseIt = request.Headers.find("X-Challenge-Response");
			if (requestChallengeResponseIt == request.Headers.end())
			{
				trace("Auth: Client did not respond to challenge");
				std::shared_ptr<Response> response = std::make_shared<Response>();
				response->Status = "401 Access Denied";
				return response;
			}
			challengeClientResponse = requestChallengeResponseIt->second;
		}

		// Compute the server version of the challenge response
		auto encryptedLocalResponse =
			Encrypt
			(
				StrToVec
				(
					std::to_string(room) +
					challengePhrase +
					challengeNonce
				),
				keyGet(room)
			);
		std::string challengeLocalResponse =
			Hash(encryptedLocalResponse.data(), encryptedLocalResponse.size());
		if (challengeClientResponse == challengeLocalResponse) // client is granted access
		{
			trace("Auth: Client challenge response matches, client authenticated");
			connVars[L"Authenticated"] = L"true";
			connVars[L"RoomNumber"] = std::to_wstring(room);
			return nullptr;
		}
		else
		{
			trace("Auth: Client challenge response does not match");
			std::shared_ptr<Response> response = std::make_shared<Response>();
			response->Status = "401 Access Denied";
			return response;
		}
	}

	// Unreachable
	//return nullptr;
}

securelib - About Those Shoulders We're Standing On

The securelib global function API provides a clean interface to the security primitive operations the application relies on:

std::string securelib::Hash(const uint8_t* data, size_t len)
{
	// https://github.com/System-Glitch/SHA256
	SHA256 hasher;
	hasher.update(data, len);

	uint8_t* digestBytes = hasher.digest();
	std::string digest = hasher.toString(digestBytes);
	delete[] digestBytes; // don't forget to delete the digest!
	
	return digest;
}

std::string securelib::Hash(const std::string& str)
{
	return Hash(reinterpret_cast<const uint8_t*>(str.c_str()), str.size());
}

//
// The trick to Blowfish encryption is that 
// you have to pad the plaintext to an 8-byte boundary,
// hence you have to tuck the original length of the plaintext into ciphertext output
// and retrieve it when decrypting.
//

inline void PadVectorTo8ths(std::vector<uint8_t>& vec)
{
	while (vec.size() % 8)
		vec.push_back(0);
}

union lenbytes
{
	uint32_t len;
	uint8_t bytes[4];
};

inline void AddLengthToVector(std::vector<uint8_t>& vec, uint32_t len)
{
	lenbytes holder;
	holder.len = htonl(len);
	vec.push_back(holder.bytes[0]);
	vec.push_back(holder.bytes[1]);
	vec.push_back(holder.bytes[2]);
	vec.push_back(holder.bytes[3]);
}

inline uint32_t RemoveLengthFromVector(std::vector<uint8_t>& vec)
{
	lenbytes holder;
	memcpy(holder.bytes, vec.data() + vec.size() - 4, 4);
	vec.resize(vec.size() - 4);
	return ntohl(holder.len);
}

std::vector<uint8_t> securelib::Encrypt(std::vector<uint8_t> data, const std::string& key)
{
	uint32_t originalInputSize = static_cast<uint32_t>(data.size());
	if (data.size() > 0)
	{
		PadVectorTo8ths(data);

		auto keyVec = StrToVec(key);
		CBlowFish enc(keyVec.data(), keyVec.size());

		enc.Encrypt(data.data(), data.size());
	}
	AddLengthToVector(data, originalInputSize);
	return data;
}

std::vector<uint8_t> securelib::Decrypt(std::vector<uint8_t> data, const std::string& key)
{
	if (data.size() < sizeof(uint32_t))
		return std::vector<uint8_t>();

	uint32_t originalInputSize = RemoveLengthFromVector(data);
	if (originalInputSize == 0)
		return data;

	auto keyVec = StrToVec(key);
	CBlowFish dec(keyVec.data(), keyVec.size());

	dec.Decrypt(data.data(), data.size());

	data.resize(originalInputSize);
	return data;
}

std::string securelib::Encrypt(const std::string& str, const std::string& key)
{
	return VecToStr(Encrypt(StrToVec(str), key));
}

std::string securelib::Decrypt(const std::string& str, const std::string& key)
{
	return VecToStr(Decrypt(StrToVec(str), key));
}

std::vector<unsigned char> securelib::StrToVec(const std::string& str)
{
	std::vector<unsigned char> vec;
	vec.resize(str.size());
	memcpy(vec.data(), str.c_str(), vec.size());
	return vec;
}

std::string securelib::VecToStr(const std::vector<unsigned char>& data)
{
	std::string retVal;
	retVal.resize(data.size());
	memcpy(const_cast<char*>(retVal.c_str()), data.data(), data.size());
	return retVal;
}

// Generate a unique string for HTTP challenging and locker key generation
std::string securelib::UniqueStr() 
{
	static std::mutex mutex;
    std::lock_guard<std::mutex> lock(mutex); // protect rand()
	std::stringstream stream;
	// https://github.com/graeme-hill/crossguid
	stream << xg::newGuid() << rand() << time(nullptr);
	return Hash(stream.str());
}

Conclusion

I hope you've seen how easy C++ REST API programming is with httplite, and how you can easily use off-the-shelf security components to build pretty secure applications, like securelocker.

History

• 6^th January, 2022: Initial version