GG-UI — Design

Home | Edit Timeline | Threading | ECS & Storage | Rendering

GG-UI is the engine's retained-mode UI library: the foundation for the editor, in-game UI, and text-first tools. It replaced Dear ImGui and ImGuizmo in the editor.

Reference pages: Core · Rendering · Interaction · Styling and Widgets. Status: see RoadmapRoadmap.


Summary

GG-UI is built from the same mechanisms as the rest of Trinyx: the spreadsheet storage model, change-frequency tiers, published frames, dirty bits, GPU-driven predicate → scan → scatter, Buffer Device Address, and lifetime tiers. It behaves and reads like every other Trinyx subsystem.

  • Zero-cost idle. Nothing is laid out, painted, or uploaded unless something changed.
  • Cost follows the change, not the document. One changed label costs one node whatever else is on screen.
  • World UI at constant CPU cost. Health bars and nameplates for thousands of entities are expanded on the GPU from slab data, and stay consistent with the displayed frame, including rollback and time travel.
  • One renderer for UI and 2D. Screen UI, world UI, and 2D sprites share one primitive format.
  • Multi-window and Wayland-first. Windows live on the main thread; docking is designed around Wayland's rules.
  • Text is first-class. Editable text buffers, a code editor, consoles, and a cell-grid mode that also renders to a real terminal.

Module Boundary

LayerLocationDepends onContains
UI coreGGUI static library (src/GGUI)A small foundation layer: math, containers, string IDs, chunked storage, Fixed32Tree and storage, mutation queue, layout, styling, interaction, accessibility rows, snapshots, primitive output. No World, Registry, renderer, or SDL
TextGGUITextFreeType, HarfBuzz, SheenBidiShaping, line breaking, the glyph atlas
Platformsrc/Runtime/PlatformSDL3Windows, input, IME, DPI, clipboard, popups, file dialogs
Render backendengine rendererVulkanThe UI pass, the texture table
Engine integrationsrc/Runtime/UIUI core + engineUIHost, the theme, the console, the inspector

The core meets the rest through three interfaces: platform events in, UI primitives out, and scheduling. The GGUI target can't include engine headers; CMake enforces it. That keeps the core headless-testable and usable by standalone tools without a World.


Trinyx Principles, Applied to UI

Engine principleHow GG-UI uses it
One data model, many consumersOne primitive format for screen UI, world UI, and sprites; one snapshot drives rendering, hit-testing, and accessibility
Spreadsheet storageNodeIndex columns, SoA rows within chunks
Tiers by change frequencyStructure and layout inputs (cold) vs computed results (published)
Published frames between threadsThe UI update publishes snapshots; Sentinel hit-tests and Encoder renders them
Dirty bitsLayout and publish queues; an idle UI touches nothing
GPU-driven compactionWorld-UI template expansion and sprites use predicate → prefix_sum → scatter
BDA everywherePrimitive buffers addressed by BDA; images by bindless index
OOP facade over a data-oriented substrateWidget functions and typed widget structs over UI storage; authors never see the layout
Lifetime tiersUI documents declare ConstructLifetime tiers; FlowManager tears them down with their domain
Determinism boundaryPresentation UI stays outside determinism and never writes sim state; sim-bound UI runs deterministically inside it

Why Not ImGui

ProblemRoot cause
Detached windows vanish or can't be interacted withImGui's viewport windows were created on Encoder while SDL events are pumped on Sentinel. SDL3 requires window calls on the main thread, and on Win32 a window's messages go to the thread that created it
Floating windows don't snapImGui docking offers drop targets only
Layout reset is unreliableDockBuilder rebuilds interact with imgui.ini restore
Immediate modeUI state is rebuilt every frame on the render thread; no place for retained structure, bindings, or GPU-side expansion

The window-thread problem isn't ImGui-specific: any multi-window UI must create and pump its windows on the main thread.

Constraints From the Engine

ConstraintConsequence
OS windows must be created and pumped on Sentinel. Wayland adds no client-side positioning, no global cursor position, and xdg_popup for menusThe platform layer lives on Sentinel. Docking and popups follow Wayland's rules
Sentinel polls at 1000 Hz and must never blockNo layout or paint on Sentinel: only routing and hit-testing against published data
Sim input is a deterministic, timestamped streamThe UI decides what it consumes on Sentinel, before anything reaches InputBuffer
The sim is authoritative and rollback-capableUI never writes sim state. It reads the published frame and acts through commands
Slab memory is owned by a World and written only by BrainUI storage isn't in the slab

Architecture

Sentinel (1000Hz)                UI update (display rate, jobs)            Encoder
─────────────────                ──────────────────────────────            ───────
Platform windows (SDL3)          Apply queued tree mutations               Read latest snapshot
Pump events                      Re-run what changed, bindings             Upload changed ranges
Route input:                     Layout                                    UI pass per window
  hit-test last snapshot   ───►  Paint                               ───►  (after the scene)
  UI takes it, or forward        Publish snapshot
  to InputBuffer (sim)
  • Tree mutations from any thread are queued and applied at the UI update: one pattern for all state mutation, like the spawn handshake.
  • Window and platform requests go to Sentinel through a non-blocking queue; results come back as events.
  • Hit-testing on Sentinel uses the last published snapshot, which is what the user sees.

Contexts and Lifetimes

A UIContext is an independent tree, storage, and snapshot chain, owned by the engine and not by a World: UI outlives World resets, spans PIE Worlds, and renders loading screens when no World exists.

UI documents declare a ConstructLifetime tier (Level / World / Session / Persistent). FlowManager destroys Level- and World-scoped UI with their domain; documents that survive a World reset get the same OnWorldTeardown / OnWorldInitialized callbacks to rebind. There is no separate UI lifetime vocabulary.

Editor Integration

The editor UI sits on the edit-mode contracts (Edit Timeline) and adds no second path into world state.

NeedMechanism
Read ECS stateThe published read frame, never Brain's write frame
Change ECS stateFieldEdit batches and EntityLifetimeCommand, applied by Brain
Undo / redoUndoHistory transactions
Interaction boundariesThe UI owns begin / commit / cancel for every interaction and opens and closes the transaction from it. Cancel restores the original
Live dragsThe widget keeps its own value while dragging, so nothing reads back a stale frame
Time travelTimeline UI drives the read-frame override; world UI follows because it is expanded from the displayed frame

Deterministic UI

GG-UI is also the basis for 2D and UI-driven games (card, puzzle, strategy), where layout and hit-testing feed gameplay and must resimulate bit-identically.

Presentation UISim-bound UI
ExamplesEditor, menus, HUD chromeCard hands and boards, UI that decides gameplay outcomes
Update rateDisplay rateThe sim's fixed step, on Brain
InputPlatform events routed on SentinelThe sim's input stream
MathfloatFixed32
Gameplay stateNone; it acts through commandsIn the slab (Temporal tier)

Commitments that can't be retrofitted, and so hold now:

  • Layout is written against a numeric type chosen per document. No hidden float in shared paths.
  • The core never reads a clock; time arrives through AdvanceTime.
  • Evaluation order is by node, never by hash-container iteration.
  • Text metrics enter sim-bound layout at one boundary, with fonts pinned per build.

Sim-bound documents on Brain are not built yet. The layout determinism test is.


Memory Model

UI storage is dedicated, built on the engine's storage principles, and not in the World slab. The slab was rejected:

IssueDetail
LifetimeThe slab belongs to a World; UI must outlive Worlds
Writer and rateBrain is the only writer, at the fixed step. UI mutates at display rate from other threads
RollbackResim would sweep presentation-only data
SizingSlab capacity is split into SystemGroup arenas; UI would need a new arena in every config
DefragMoves EntityCacheIndex; UI handles would need rehydration

Each UIContext owns chunks of 64 slots, allocated on demand and freed when empty, like archetype cold storage. No fixed capacity, no per-node heap allocation, and nodes never move.

  • Handles are a stable index plus a generation. Freed slots are reused lowest-first to keep chunks dense.
  • Rows are SoA within a chunk, grouped by how often they change.
  • Sparse rows (NodeSparse<T>) hold what few nodes have: a node without a value costs four bytes.
  • Per-node cost: 483 bytes including its share of four snapshots.

World UI and 2D

Not built. The primitive format and the binding model are fixed so these can be added without touching the core.

World UI is expanded on the GPU with the predicate → prefix_sum → scatter pattern:

  1. Template: a primitive list in local space plus bindings driven by entity fields through a fixed set of GPU-evaluable ops (Core).
  2. Entity side: a Volatile component (CWorldWidget { TemplateID, AnchorOffset, Flags }).
  3. Predicate: active + has widget + frustum + distance.
  4. Prefix sum over visible entities × primitive counts.
  5. Expand into the UI primitive buffer, using the interpolated positions scatter produced, or widgets jitter against meshes.

Consequences: constant CPU cost; always consistent with the displayed frame, including rollback, time travel, and undo. Arbitrary strings are shaped once on the CPU and placed by the GPU. Transient world UI (damage numbers) goes through TemporalEvents and the PresentationReconciler, so mispredicted numbers fade with an Anti-Event.

2D is the same path as a sprite renderer: sprite entities instanced from the slab, an orthographic camera, sort layers.

UI renders after scene reconstruction at output resolution. It must never pass through temporal reconstruction or upscaling, which is what smears text.

Text, Code, and Terminal

  • Text buffers and editing are built (Interaction).
  • Code editor (planned): incremental tokenizer, LSP client (clangd), diagnostics, folding, breakpoint gutter.
  • Consoles are built (Interaction).
  • Cell-grid mode and a terminal backend (planned): cell-grid documents rendered to ANSI/VT with no GPU, so dedicated servers and CI get a live console from the same documents. This is why the core emits renderer-agnostic primitives.

Code ↔ Node Graph

The node graph and C++ are two views of one program. The C++ source is the source of truth; the graph is a live projection of it.

  • Code changes re-parse incrementally and update the graph, keeping each node's identity through its source anchor.
  • Graph changes become minimal text edits applied as one transaction, so undo covers both views and comments and formatting survive.
  • Today's paths are one-way and lossy. The round-trip model gets its own design document.

Platform Layer

On Sentinel, over SDL3, behind an engine-owned interface: no header above the platform layer includes SDL. SDL3 already handles the hard Wayland surface (fractional scale, xdg_popup, text-input-v3), and native backends would cost months for parity.

  • Multiple windows, one swapchain each.
  • UI units. A window's UI scale is its monitor's display scale × the user setting (UIScale, --ui-scale). Layout and text re-rasterise at a new scale; nothing is bitmap-stretched.
  • Per-window input routing, text input and IME, clipboard (including the X11 / Wayland primary selection), system cursors, drop receive, popup windows, file dialogs, window ownership (keep-above). All asynchronous: a request from any thread, the answer as an event.
  • Closing a window closes its popups first, so the renderer releases every surface before the OS window goes.

Native code behind the same interface:

GapPlatformStatus
Custom title bar with Snap LayoutsWin32Done. The app publishes its maximize button's rect (SetMaximizeRegion) and it hit-tests as the OS button
Modal move / resize loop stalls SentinelWin32Done. The platform ticks Sentinel about 60 times a second during the loop (SetModalLoopTick). The OS still waits up to ~0.5 s on a title-bar press before entering it
Starting a drag between windowsWaylandPlanned: a wl_data_device source of our own (Docking Under WaylandDocking Under Wayland)
IME candidate window placementAllSetTextInputArea from the focused widget's caret; unverified per IME

Linux Verification

Built and checked on Windows. Everything goes through SDL and is expected to work, but X11 and Wayland runs are incomplete. Use the platform test window (--platform-test); its controls are listed in PlatformProbe.h. The routing logic (PlatformRouter) has no OS dependency and is covered by headless tests on every platform.

AreaKnown platform differences
WindowsWayland: the compositor places top-level windows; SetPosition does nothing
Custom chromeWayland: no Snap Layouts equivalent. Standard windows need client-side decorations (libdecor) on GNOME
PopupsWayland: xdg_popup; the compositor may reposition or dismiss one, and a menu must open in response to input
Keyboard focusWayland: raising a window needs an activation token and may be refused
ScaleWayland: fractional scale, so logical and physical pointer positions differ. X11: one global scale
ClipboardThe primary selection exists only on X11 and Wayland

Docking Under Wayland

CapabilityWin32 / X11Wayland
Dock inside a windowYesYes
Tear off into a new windowYesYes (the compositor places it)
Layout save / restoreYesYes
Drag a tab between windowsYesNot yet: needs a drag source SDL doesn't expose (planned below)

Capabilities are detected at run time.

Between windows, a tab drag finds the window under the pointer from window positions (UIHost::DockDocumentAt). Wayland gives neither window positions nor a global pointer, and during the press the source window keeps every pointer event. Only the compositor knows which surface is under the pointer, and it says so only during a wl_data_device drag. SDL 3.5 receives those drags but can't start one.

The plan, from reading SDL's Wayland backend; to be built and checked on Linux:

  • Source. The platform layer binds its own wl_data_device_manager and wl_seat on SDL's wl_display (SDL_PROP_GLOBAL_VIDEO_WAYLAND_WL_DISPLAY_POINTER). They share SDL's default queue, so SDL's event pump dispatches them. A wl_pointer of its own on that seat receives the same button events, which carry the serial start_drag needs. SDL keeps that serial private.
  • Start. When a tab leaves its window, start a drag from that window's wl_surface with a source offering a private text MIME type and a token naming the drag. Optionally show a drag icon of the tab.
  • Target. SDL accepts text drops, so its own data device turns the drag into SDL_EVENT_DROP_BEGIN, DROP_POSITION (window-local), DROP_TEXT and DROP_COMPLETE on whichever of our windows is under the pointer. These already reach the platform router; a drop carrying our token is routed to CarryOver / CarryEnd instead of a text drop.
  • Outside. The source's cancelled event means nothing took it: float the panel. The compositor places it.
  • Problems. Starting the drag sends wl_pointer.leave to the source window, and SDL answers that with a synthetic button release. The platform layer must swallow it, or the drag ends where it started. Another application may accept the text type; send writes nothing, so it receives an empty drop. Links wayland-client directly; SDL loads the same library at run time.

Roadmap

#MilestoneStatus
M1Platform layer: multi-window, input routing, text input, DPIDone on Windows; Linux verification open
M2UI core: storage, tree, mutation, layout, snapshots, accessibility rows, determinism testDone
M3Rendering: primitive pass, clipping, bindless images, textDone
M4Interaction: routing, focus, scrolling, text editing, popups, string tables, console, inspectorDone
M5Styling: widget definitions, tokens, rules, transitions, sheets with hot reload, the engine themeDone
M5b–cCore at scale: incremental publish and paint, documents on jobs, late documents leaving the frameDone
M6Editor widgets: virtualised lists, trees, tables, tiles, property grid, panels, drag and drop, input actionsDone
M7Docking and multi-windowDone on Windows; step 6 to do on Linux
M8Editor migration, panel by panel; engine-rendered gizmo; engine FieldSource and string tableDone
M8bNode graph canvas; code editorGraph done (Core); code editor open
M9Game UI: game state in code, widget assets, the HUD elements, rich text and action prompts, style scopes, animation, starter widgets, the designer, configurable input, GPU world-UI templates and screen markers, sprites, sim-bound documents (Widgets)Designed
M10Remove ImGui and ImGuizmoDone
M11Accessibility bridge (UI Automation, AT-SPI)
M12Visual logic in widget assets
M13Terminal backend
M14Code ↔ graph two-way sync

M7: Docking and Windows

StepWhatStatus
1The dock tree in one window: splits, tab groups, tabs dragged to rearrange, panels closed and reopenedBuilt (Core)
2Named layouts, the default always available, saved as textBuilt
3Panels in their own OS windows: tear off, dock back, drag between windows, keep-aboveBuilt
4Dialogs, a message with choices, notices, the platform file dialogBuilt (Core)
5Menu-bar hover and arrow switching; a toolbar overflow menu; more tabs than fitBuilt (Core)
6Dragging a tab between windows on WaylandTo do on Linux (planplan)

Untested by hand: the real file dialog, tab dragging with a real mouse, and layout restore across a restart. Nothing after the first Linux pass has been built on Linux.

M8: The Editor on GG-UI

The shell moved first, so there was one dock from the start (EditorShell); panels not yet moved over were drawn by ImGui into areas of it until M10. The editor waits for every document each frame (an update budget of 0) and its frame on Encoder reads what was just published; the shell's handlers post their work to that frame.

Shortcuts are rebindable actions under Editor.*, with defaults most editors share:

ActionDefault
Open, save, save as, save allCtrl+O, Ctrl+S, Ctrl+Shift+S, Ctrl+Alt+S
Undo, redoCtrl+Z, Ctrl+Y (or Ctrl+Shift+Z)
Duplicate, deleteCtrl+D, Delete
Select (no gizmo), move, rotate, scale; cycle move / rotate / scaleQ, W, E, R; Space
World or local spaceX
Frame the selection; drop it to the floorF; End
Maximize the panel under the pointerShift+Space
Play (or go on playing); step a paused sessionF5; , and .
Stop playing; give the pointer and keys back to the editor while playingEscape; Shift+F1
Fly the camera (held with the right button)W, A, S, D, E, Q

While playing, the game has the keyboard, except chords of actions kept from the game (UIHost::KeepFromGame): Stop and Release Pointer. Their commands exist only while playing, so Escape keeps its usual meaning otherwise. Keys that also fly the camera (W, E, Q) don't switch tools while it flies.

Open: hiding entities and Constructs in the scene, so a designer can place things that a trigger reveals. It needs its own design: whether a hidden entity collides, ticks, replicates, and can be picked in the editor.

StepWhatStatus
1The shell: title bar with menus, workspaces and play controls, the dock, editor actions (Editor.*) in Keybinds.iniBuilt
2EntityFields (a FieldSource over an entity's reflected components: reads the published frame, edits as FieldEdit, one undo step per interaction); TnxStringTable (a file per language, reloaded on change) and an interface language settingBuilt (Core, Interaction)
3The scene viewport as a ui.Viewport (picking and camera fly through its pointer events); ui.Canvas (lines, polylines, arcs, triangles) and the gizmo drawn on it over the viewport, replacing ImGuizmo; the grid drawn in the scene with depth, as a ground plane to judge size byBuilt (Rendering, editor)
4Panels: Log, Engine Stats, Outliner (a tree showing parenting), Details, Content Browser, DebuggerBuilt (editor); parenting waits for attachments in the engine
5Dialogs and overlays: open / save, import, unsaved changes, asset issues, prefab save, the command palette, the frame budget stripBuilt (editor); file choices use the platform dialog, the budget strip became the status bar
6Play in editor: the local game in the viewport; Authority and Owner views as dock panelsBuilt (editor)
7Construct, Entity and Prefab editors; the component generatorBuilt (editor); their node graphs moved over in M8b

String files are hand-editable, one per language, so anyone can add a language:

# content/strings/en.strings
Editor.Outliner.Title = Outliner
Editor.Scene.Open.Tip = Open a .tnxscene file

Not Placed

  • A first frame in proportion: build, layout, and publish of a large document from scratch cost about 0.7 microseconds a node, and need the three phases to walk chunks rather than nodes.
  • Charts.
  • A full engine input action (TnxInputAction); see Interaction.

Performance

Measured on one core, optimised build, fixed-metric text. GGUI_Scale holds every run to these; Performance Targets has the full list.

For 100,000 nodes
An update with nothing changed0
One changed label: build, layout0.0024 ms, 0.0019 ms
One changed label: publish0.015 ms
Hit-test0.008 ms
Finding a node in a snapshotunder 0.0001 ms
Build from scratch26.5 ms
A list scrolled slightly, 200 nodes showing0.015 ms, no rows written
Paint
One changed label with 40,000 primitives showing0.0017 ms
10,000 primitives from scratch0.20 ms
Widgets
A list of a million rowsBuilt in 0.28 ms; 0.003 ms per frame scrolled within a row
A table of a million rows, six columnsBuilt in 0.83 ms; 0.011 ms per frame scrolled within a row
A late document (250 ms update)The other documents' frames took at most 0.05 ms each
Text shaping, eight threads1,800 strings in 9 ms (16 ms on one thread)

How It Got There

Design choices that produced those numbers, and the problems that forced them:

  • Snapshots share unchanged row chunks. Publishing one changed label used to write every node (9.7 ms at 100,000 nodes).
  • Rows hold unscrolled positions; scroll offsets are stored separately. Scrolling a box used to rewrite the row of everything inside it (20 ms for a 100,000-node list).
  • Nodes keep their primitive slots; draw order is a separate list. A label that grew by one glyph used to move every primitive drawn after it.
  • Each document caches its own resolved styles (GGLooks), so a style table is only read during updates and documents sharing one update in parallel.
  • The host never touches a document that is updating, and never changes what one reads. A slow document holds one frame, the one that discovers it, and none after.
  • The text engine is three things, not one lock: a run table read by many, shaping with per-thread FreeType and HarfBuzz objects, and an atlas locked only to place a new glyph.

Problems found along the way, outside the UI:

  • Job system: a quarter of the workers take physics jobs only, and a submitted job's wake went to any sleeping worker, so a job could wake one that couldn't take it and sit until something else was submitted. A coordinator that helps with its own jobs hid this. Physics-only workers now wait on their own signal.
  • SheenBidi: its lock-free scratch-buffer stack handed one buffer to two threads under contention and crashed four runs in ten once shaping was parallel. It is built without that cache.
  • Store forwarding: primitives written a field at a time and copied straight after stalled the copy; see Rendering.

Decision Log

DecisionChoiceWhy
LibraryCustom, engine-nativeBuilt from Trinyx's own mechanisms; one foundation for editor and game
Module boundaryA core library with no engine dependency; platform, renderer, and integration as engine layersHeadless tests, standalone tools, replaceable layers
MemoryDedicated chunked storage per UIContextLifetime, writer and rate, rollback, sizing
LifetimesConstructLifetime tiersOne lifetime vocabulary across the engine
World UI instance dataIn the entity slab, expanded on the GPUConstant CPU cost, consistent with the displayed frame
Frame positionAfter scene reconstruction, at output resolutionText must never pass through TAA or upscaling
ThreadingSentinel routes; the UI update runs on jobs and publishes; Encoder rendersSentinel never blocks; windows stay on the main thread
TextFreeType + HarfBuzz + BiDi from the startShaping can't be retrofitted cheaply
AuthoringA C++ builder; widget assets as data built at runtime through itEditor needs came first; assets give hot reload and the designer one format that already runs
Game data in UIAssets declare typed contexts and read their fields from the published frame; game code supplies the contextsNo per-frame binding code; right across rollback and time travel
WindowingSDL3 behind an engine-owned interface, with native code for known gapsWayland support; replaceable per subsystem
Editor reads and writesRead the published frame; write through FieldEdit inside undo transactionsBrain stays the only slab writer; undo is one mechanism
DockingOne host-owned dock space; floating panels are always OS windowsThe platform's snapping applies, and it is the only kind Wayland lets be placed
Notices and file dialogsServices of the host, reached through handlesA window isn't a world's; a handler has no ui
StringsString IDs with a default table, or source text as the keyIDs survive rewording; plain text needs nothing declared
DeterminismTwo document modes: presentation and sim-boundUI-driven games need bit-identical resimulation; presentation UI shouldn't pay for it
Code ↔ graphC++ is the source of truth; the graph is a projectionTwo-way editing without losing hand-written code

Open Questions

  • The op set for world-UI template bindings: what is the minimum useful set?
  • The accessibility bridge: build on an existing library (AccessKit) or implement the platform APIs directly.
  • How UI code touches engine state safely from a document updating on a job.