Apple researchers have unveiled SimpleDesign, a new AI model that jointly generates protein sequences and continuous 3D structures. Detailed in a newly published study, the model skips traditional multi-stage training pipelines in favor of a single end-to-end process.

Last September, the team released a study titled "SimpleFold: Folding Proteins is Simpler than You Think," detailing a model that predicts a 3D structure from an amino acid sequence. This involves predicting precise 3D coordinates based solely on a string of characters with 20 possible values. SimpleFold dispensed with complex architectures by relying on standard transformer blocks and can be efficiently run locally on a Mac with Apple silicon using MLX. Now, the researchers are applying that push toward simpler architectures to the broader challenge of multi-modal protein co-design. Grasping this multi-modal relationship is considered crucial for fields like drug discovery and protein engineering.

Bypassing the multi-stage bottleneck

Many existing protein design models rely on a multi-stage process. They typically train an autoencoder to convert structures into discrete tokens, and then train a second generative model on those latent representations. This introduces specialized tokenization models and structural training stages that can complicate pipelines with unnecessary overhead.

SimpleDesign hypothesizes that this intermediate step is unnecessary to achieve performant co-design. The model skips compressing protein structures into a tokenized representation, acting directly in the data space. It leverages a single-stage end-to-end objective that combines discrete cross-entropy for processing sequences and a regression objective for handling structures. To model these differences effectively, the framework utilizes Transformer-based multimodal backbones.

Training on two million proteins

To build SimpleDesign, Apple researchers utilized more than 2 million protein sequence-and-structure pairs. These were primarily drawn from the AFESM dataset, which combines predicted structures from the AlphaFold Database with additional samples.

During training, researchers deliberately corrupted both parts of each data pair. Amino acids were randomly hidden behind masked tokens, while the corresponding 3D structures received added noise. By varying the scrambling levels, the model learned multiple tasks simultaneously.

If the structure was heavily corrupted but the sequence remained mostly intact, it functioned like protein folding. Conversely, hiding the sequence while leaving the structure intact trained the system for inverse folding. By scrambling both simultaneously, the model was effectively trained for full protein co-design.

Benchmarks and physical limitations

According to the study, SimpleDesign achieved competitive results across structure generation, sequence generation, and protein co-design benchmarks. The researchers noted that its generated amino acid sequences matched or exceeded those from most competing multimodal models.

However, the team acknowledged a current limitation with the research. The results remain confined to computer-based evaluations. The generated proteins have not yet been experimentally tested to confirm they fold, function, or behave safely in actual biological systems.