AbstractPhila PRO

Seems we've run into engineering problems. The time degraded from 4 seconds per step to 9 seconds per step, compounding increase per independent arm attached.

I need to halt and run analysis. As it stands, we'll run until the end of Friday, then the engineering goes up on blocks.

I'm going to squeeze every bit of every byte from this engineering. Formulas up for scrutiny, system optimizations up for scrutiny, kernel optimizations, libraries, everything. Running until October 21st isn't reasonable. The models must be prepared correctly and reasonably to the mathematics, the results must be calculated correctly and the model trained to parity without faulty modifications during training that can impact the results.

Here's an interactive viewer for the internals of Mini-Beatrix-2.5s

I'll enhance it for 3 when it's ready.

https://claude.ai/artifact/YH3BfrBWiPM5QfsgDuuKuJ

As a comparison to global erank, we're looking at a structure of 400+ for around half of Beatrix V3 so far, so roughly 16+ blocks of erank >400, substantially stronger than the original two models for geometric attribution. The final block has a collapsing problem currently, but I believe others have the answer with autoregression models through a finalized projection smoothing layer concept. I haven't employed it yet though.

The fractal instability hits pretty early. You need rounding structures early otherwise the gradients explode at one point or another. The predominant problem was loss explosions. It happened because of ill-formed eigens in the intentional step structure I was experimenting with. 5 step cantor essentially ensured the cantor fractals deviate to a certain degree, and depth itself was meant to raise the steps of fractals to new states and interpolate the fractals.

If you use any of this, make sure you either pass it into AI for optimization - as it's likely terrible due to being my earlier models (I came from game development, optimization is very different). AI will be able to improve the speed and accuracy of the formulas.

https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/core/vit_beatrix.py
https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/positional/cantor.py
https://github.com/AbstractEyes/geofractal/blob/main/src/geofractal/model/core/geo_fractal_david.py

One of the problems was similarity. Almost everything was self similar, which in theory should have helped differentiate. However in practice, the structure found it's own similarity attractor basins that cause cascade corruption down the chain. The only solidity was to introduce eigen comparators through decomposition learning, which is a little different than autoregression. With this, the decomposition required more accuracy otherwise the system would always default to 1 of the first 3 steps - resulting in rigid or slightly less rigid articulations.

I measured fp64 being required for stable 4 step, and fp roughly 92 to be in a safe zone for stable Mandels at step 5. Julia requires something substantially larger than mandels. Fp64 is ENOUGH for rotary offset in standard positional systems, however fp128 is required for something akin to cantor fractal positional systems of differentiation.

It happens due to the eigenvalues themselves often malforming, and the subsystem silently rounds them. Using FULL SVD is a compositional fix for comparison, with that introduces a huge overhead as well.

Fractals themselves turned out to be more compositionally useful, not as additive elements, but as miniature rounding structures. The splat there was built under the concept of eigen substitution, meant to composite a series of tiny opinions from tons of subsystem residuals together into a composite "blackboard", forming a more robust and structural aligned INK BLOT splat, similar conceptually to viewing a random inkblot. This eventually composites into a utility of structural awareness, and it really doesn't take very long.

Essentially, that structure is geometric in nature, but it's not using Eigenvalues directly. It CAN use them, it should be capable of using any structural bounds with attributable contributions.

Splat functions viably at bf16, is a bit slower than MHA, but houses geometry more cleanly than MHA (sometimes by a huge margin) when trained with MUON instead of adam, adamw, or another multitude of optimizers I ran. I have attempted custom optimizers to encourage this behavior further, but the results showed MUON is just better.

Give it a shot in something simple, it'll train fast enough.

Pretty much anything in here is useful.
https://huggingface.co/collections/AbstractPhil/geolip-research-concepts

This is the schedule for the v3.

Eigens and causal chains have correlations but not causation without additional contributions to the assessments, the SVAE shows this to be a guarantee in many shapes, and in many others impossible.

The accuracy between the two requires a smoothing system, alpha differentiation through projected MHA-esque alpha attention to patchworks in order to fill the gaps. They don't directly line up quickly though, it looks more soupy when it's done.

They coalesce, but the extractions aren't consistent enough to directly use without a series of wrappers and structural alignment systems. Cantor Aleph and Omegas are essentially this structural system, but they are unstable. Cantor fractals remain unstable until around fp128 for Mandelbrot without redefining the underlying methods the mathematics linalg system uses. I did some headway on this, but I ran into a glacier that I would have needed to sink months into to make headway so I built a system to replace the slower linalg systems and the system lost much of it's cantor fractal capacity in favor of reproducibility and consistency.

The prototype forged from a 52,000 battery sweeps to find the most consistent recon convergence over time, heavily scrutinized and analyzed for over a month to build into something useful.
https://huggingface.co/AbstractPhil/geolip-SVAE

The current best case of the eigens research conclusions. Everything SVAE built to the attention prototype, everything constellation built to the processing and lookups for the model, everything distillation built to the banks for capturing and yielding, everything structural built from the knowledge and wisdom of other researchers. Well, not everything structural - it needed a lot of geometric formation and structural cohesion to make it work.
https://huggingface.co/AbstractPhil/aleph-splat-0/blob/main/splat_attention.py

Attempts to speed the SVD up were somewhat fruitful, somewhat not. They are good for inference, but I never programmed the gradient backprops for it.
https://huggingface.co/AbstractPhil/svd-triton

The mobius lens being a faster form wasn't strong enough as an activation system. I needed an architecture around it, not just an activation.
https://huggingface.co/AbstractPhil/mobiusnet-distillations/blob/main/make_chart_1.py

The bytelex atlas has yielded massive results so far. Beatrix V3 will provide all the necessary implications for a massive multi-spectra multimodal distributed distillation.

We're almost there.

After a week of faulty and failures using generic structures with bytelex, I found a successful aleph prototypical structure that conforms to the needs.

• We've managed to overlap multiple arms to train multiple simultaneous templates.
• Introduce new tokens.
• Retrain existing tokens.
• Extend new chains and new behaviors from training.
• Properly instantiate and reinforce behavior using Aleph RNN.

The EMA Relay. The code has been pushed to both beatrix repos.

The structure itself is built specifically as a solidification unit to extensible arms, allowing more composite structures to build.

EMA structures aren't new, but when applied correctly at just such a methodology, the models begin to behave as though the extension relays are in fact the original model. The chains and behavior form naturally and the substructure begins to conform with the token fragments from much more complex structures like combined token differences of T5, Qwen, and CLIP as unified teachers.

The cross-token noise is mitigated using a series of principles and the blueprints are showing both solidity and failure simultaneously, both proving many new utilizable states and disproving multiple theoretical pathologies utilized in current running modern papers as the methodologies tested in the specific formats.

60 hour battery under way currently. Currently up to around three sentences or so of bytelex capacity with multi-tokenizer inference comparisons.

Not the strongest yet, however the validation and test cases are showing promise at between 60 and 80% at highs with the canary recall remaining at around 97%, lows completely collapsed for multiple experiments.

Heavy experimentation with GRU, RNN, and multiple other components to test standard component utility.

So far so good. Many prototypes establishing information from many byte structured distillation routes.

I release everything MIT, but you can't find the trained weights elsewhere. It's too much data and too much space to host reasonably elsewhere today. If a rival crops up I'll dual-host most likely.

The control version essentially imploded with the same data and same shape. SDPA couldn't... actually represent the data. The erank completely collapsed and the model density is essentially multi-stage collapsed during the curriculum.

SDPA in every block instead of splat, failed...??

I legitimately didn't expect that. It will be in the article. I need to revise some information. Almost every core and key test showed the SDPA could potentially overtake the splat, but the actual outcome was a direct contradiction.

The collapse symptoms were showing early stage shape similarity to the beatrix-v1 splat + sdpa hub combo pack, which I did not expect what-so-ever. I expected the model to recover and form possibly more strength over time in each layer. The belly of the model bloated, then once the curriculum hit, the model turned inside out within 5000 steps. The model went from semi-functionally pretrained showing potential weaknesses in early bottleneck stages, leading to later layer structural boundary collection as the v1 did, and in the later training she completely collapsed.

The structure began collapsing during the curriculum that trained the v2 splat variant with some overfitting, but not collapse.