US Patent 11586832 Parallel unicode tokenization in a distributed network environment

Unicode data can be protected in a distributed tokenization environment. Data to be tokenized can be accessed or received by a security server, which instantiates a number of tokenization pipelines for parallel tokenization of the data. Unicode token tables are accessed by the security server, and each tokenization pipeline uses the accessed token tables to tokenization a portion of the data. Each tokenization pipeline performs a set of encoding or tokenization operations in parallel and based at least in part on a value received from another tokenization pipeline. The outputs of the tokenization pipelines are combined, producing tokenized data, which can be provided to a remote computing system for storage or processing.