An Improved Hybrid Encryption Scheme Based on the Sequence of Reduced Residue Systems

Abstract

The rapid growth of digital communication has created an urgent demand for advanced cryptographic techniques that ensure both security and efficiency. As reliance on digital platforms increases, so does the risk of cyber threats and data breaches. Robust cryptography is, therefore, essential to protect sensitive information and maintain trust in digital transactions. This research proposes an improved composed hybrid cryptosystem that integrates Transposition, Caesar, and Hill ciphers, followed sequentially by RSA encryption. The study examines how a hybrid of four ciphers can be attacked when treated as a composite function. To further enhance security, a sequence of Reduced Residue System (RRS) values was introduced to replace ASCII characters after the third cipher (Hill cipher), adding an additional layer of residue-based encryption before the final RSA stage. The findings demonstrate that the improved hybrid cryptosystem significantly enhances data security and key generation efficiency, adding a new level of complexity that makes it more challenging for attackers to guess or compute decryption keys.