Jun: Ticks, Netcode & Traces!
Ticks, Netcode & Traces!
Welcome to our roundup of the latest updates from the last month!
Ticking
Ticking is the beating heart of a game, it's where we process inputs, network callbacks, scripting events, and everything else.
We've massively improved our tick loop and our Timer system, on both sides:
Server-Side
We completely reworked the core tick loop, by implementing a more precise timing algorithm. This new algorithm eliminates the small but cumulative timing errors that used to affect internal timers and even the Timer class.
With that, we changed the Config.toml config from tick_rate to max_tick_rate, now in Hertz (Hz), instead of Milliseconds. The default value is now 30 Hz (i.e. 33.33ms per frame), matching the client side for better synchronization.
We also improved the logs for High Tick durations, displaying better diagnostics like this:
WARNING Server Tick time extremely high! Server may be very overloaded or stuck.
Tick Duration: 117.90ms (8/s). Average: 39.07ms (25/s). Configured: 33.33ms (30/s).
Client Side
Previously, all nanos world processing happened after physics simulation. This included networking, snapshots, interpolation, Lua event handling, CEF painting, 3D Debug Drawing, etc.
It worked... but we realized it could be way better.
We've now reordered and prioritized internal ticks so low-latency tasks (like networking sync and inputs) happen ASAP in the frame and things that need the frame to be fully read executes after we have the data ready to be consumed.
This makes sure the correct logic happens when the frame is ready for that, fixing having on "old" frame data to work with.
The result? A dramatic boost in perceived smoothness and responsiveness. Here's a before-and-after comparison using the PhysicsGun:
Sandbox PhysicsGun before, with lots of jittering and desync on the texts and Particle location
Sandbox PhysicsGun after, with no jittering and ghosting anymore
This huge improvement just by adjusting the order of the algorithms are processed.
Timers
Timers previously relied on delta times between frames. Over time, this caused slight inaccuracies, it was really subtle, but enough to accumulate.
We now use a real-clock-based algorithm to track elapsed time with far greater accuracy.
It's really noticeable when comparing Timers and the Lua clock methods, they being nearly perfectly in sync.
Timers and Tick events still inherit the margin of error from the configured frametime (e.g. 33ms).
We also fixed a long-standing bug in Interval Timers, which weren't compensating for execution overhead. Now, a timer set to fire every 1 second will fire every 1 second exactly, no matter what and with no delays in sequential calls.
Netcode Improvements
Also, thanks to the Ticking improvements, and other internal low level networking improvements, our network latency has dropped significantly! The quality feeling of playing with someone on the other side of the world increased dramatically!
We even lowered the threshold for Debug Drawing colors to detect delays more accurately. Here you can notice how better and evenly spaced are the packets being received now:
Network Snapshot Packets being received before: too big corrections (big arrows) and delays (yellow)
Network Snapshot Packets being received now: evenly spaced, mostly green and small
The bigger the arrow, the bigger the "error" from the local interpolation and the received network real location.
Also yellow arrows means a packet too more than 2 frames (66ms) to be received. This threshold was reduced from the old value 450ms, and even so we barely see them yellow now!
Read more about Network Debugging and what each color/arrow means at:
🧑💻 Authority Conceptscore-concepts/scripting/authority-conceptsTraces
Some great improvements on internal Traces handling.
Trace Visibility
Our trace functions used Unreal's TraceByObjectTypes, which doesn't care about the collision trace channels. That worked fine for nanos world entities and custom created maps, that were created with gameplay purposes in mind.
But we found that some user-made maps (or FAB maps) had objects (like WorldStatic) set to ignore collisions on the Visibility channel, but with Query Collisions still enabled. These objects were still being detected by our traces, even when they shouldn't be, breaking some aspects of the gameplay, like Sandbox's PhysicsGun or Spawn Menu not working properly.
To fix this, we added a new TraceMode.TraceOnlyVisibility to the TraceMode enum.
This new TraceMode ensures the traces only return objects that match the object types passed and have the Visibility channel blocking, making it a must for our Sandbox common traces and fixing all the issues we were having with custom FAB maps.
Impact Point
As a request from the community, we are also adding a new information to our Trace returns: ImpactPoint, this is the contact point in which a trace collided with the impacted object!
Asset Pack Auto Reload
A frequent issue in development environments was that developers could modify the Asset files while the server was still running, making all the precomputed hashes technically invalid, causing clients to mismatch local files, and in some cases, request newer files from the server using outdated hashes, resulting in download errors and preventing them from connecting until a full server restart is made.
We're introducing a new Asset Pack Auto Reload system! This new system detects real-time file changes inside asset folders and queues an automatic reload of the affected Asset Pack. It recalculates all hashes (just like on server start), allowing updates to Asset Packs while the server remains online.
This is not a Hot Download system. Connected players will still need to reconnect to download the updated files.
This system aims to eliminate many problems during development, such as inconsistent hashes, silent desyncs, or mysterious crashes.
While an Asset Pack is reloading, new player connections will be temporarily blocked until it's fully ready.
Loading Screen Data
We've revised the structure of the LoadingScreen JavaScript object. Now, server and player information are neatly split:
var LoadingScreen = {
server: {
ip,
port,
name,
description,
},
player: {
nanos_id,
nanos_username,
steam_id,
}
}
The old format is now deprecated and will show warnings if used.
New System Stats
We've added a small but useful debug feature: System Stats. It appears alongside the Frame Timings panel when enabled and shows real-time RAM and VRAM usage.
You can enable it in the Debug Settings!
We intend to expand this stats to display more information in the future.
Changelog Markdown
We've added a new in-game Markdown parser that converts Markdown into Unreal Rich Text. This means changelogs are now properly formatted and beautifully displayed inside the menu:
Game Menu Changelog Widget displaying header and bold texts
No more plain text or weird broken tags!
Fixed River
It's becoming a bit of a tradition: from time to time, our rivers (or water in general) break completely. 😅
For quite a while, a visual bug caused rivers to appear totally broken unless you manually saved the settings. This affected every river on every map, and until now, we couldn't find a reliable fix.
But we managed to implement a workaround to mitigate the issue! So rivers can flow properly again without the glitchy rendering gaps.
UI Responsiveness
We fixed several UI screens that were breaking in non-16:9 aspect ratios (like 4:3 or 16:10). The Vault, Settings, Find Servers, New Game and more are all now fixed!
Everything should now scale correctly across common screen formats:
Vault Screen displaying properly on 4:3
Vault Screen displaying properly on 16:9
Other Improvements
File Transfer Overhaul
We've made multiple improvements to the file transfer system, again! We've drastically revamped our internal systems to improve the parallel downloading, fixing all the stutter/hitches happening while downloading each file.
We also improved the robustness of the system as a whole, to prevent crashes and improve the stability of the downloads!
Lua Internal Stack
We've made a structural improvement to our Lua implementation to ensure that API calls now behave more like native Lua function calls. Now the Lua stack is preserved during API calls, as before it was always being cleared. While this didn't cause major issues before, it prevented tools like hooks from accessing correct stack information on API calls.
Related Issue#932
Lua Internal Memory
We've enhanced Lua's internal memory management, addressing issues where the garbage collector failed to properly clean up the Environment and some Userdata after they were unloaded, resulting in potential memory leaks.
All entities and environments are now correctly flushed from memory, resolving problems that previously caused Files and Databases to remain open, and even crashes.
Firewall Alert
Windows Firewall Popup
We've improved Steam installation process to auto add all executables automatically to Firewall when installing nanos world. No more popups asking your permissions when opening special WebUI pages or launching the server!
Fatal Errors
We fixed a lot of Fatal Errors of Failed to send Message that were annoying us for some long time due an internal bug. They happened in both Client and Server and were flooding our logs and our crash reporter.
Conclusion
This was another month of polish and core improvements, I've been fully focused on making nanos world and all its internal systems as stable as possible, aiming to a state with almost no crashes or chronic issues. This foundation is essential before diving deep into major new features and being able to focus on them without distractions.
These improvements are crucial for building a solid structure for the game: smooth gameplay, consistent behavior, and fewer bugs. That reliability is key to getting the trust of both players and developers to keep working and creating awesome content on it!
More GitHub issues were resolved, and there are still a few more I plan to address before I can consider this round of housekeeping complete. Our new internal crash-handling implementation have already proven really effective, it gives me valuable insights to solve some rare errors and crashes. If you're experiencing a crash and know how to reproduce it, please let us know! This speeds up a LOT the time to fix them!
There are already many exciting things being worked on behind the scenes, and I've been receiving great support that will allow me to bring amazing updates very soon.
Thank you everyone for more this month of tests and polishing! Also, thank you to everyone who donated to my Ko-fi, your support means a lot and helps keep this going! See you next month!! 😄
