This paper presents a complete EEG encryption framework based on a newly designed fractional variable-order memristive hyperchaotic system. The proposed system exhibits high-dimensional nonlinear dynamics with tunable memory effects, making it highly suitable for cryptographic operations requiring strong sensitivity, unpredictability, and large keyspace. A nonlinear keystream generator is constructed using chaotic state mixing, rotate-and-diffuse Boolean operations, and multi-level quantization to ensure statistical balance and high entropy. The keystream drives a lightweight byte-wise XOR encryption architecture for multi-channel EEG signals. To validate the proposed scheme, an extensive experimental pipeline is implemented, including eight security analysis figures: keystream distribution, autocorrelation, power spectral analysis, EEG autocorrelation, cross-correlation between plain and cipher EEG, and comparative PSD evaluation. Additionally, NIST randomness tests (Frequency, Runs, Spectral, Approximate Entropy) and PRD distortion analysis across 20 EEG channels are performed. Results confirm that the proposed fractional variable-order system generates highly random keystreams that pass all NIST statistical tests and provide near-zero reconstruction distortion. The results show that the proposed scheme strengthens the overall security of the encrypted signals far beyond what is typically achieved with conventional chaos-based approaches. Experimental results indicate, Entropy = 7.9188 bits, NPCR =99.57%, UACI = 33.656%, Correlation =-0.013968. The generated keystream successfully passes all employed NIST tests. PRD values remain below 1% for all twenty EEG channels. These findings demonstrate excellent randomness, diffusion capability and negligible reconstruction distortion.