Commit Graph
100 Commits
Author SHA1 Message Date
Grant Limberg c02c649cd3 more build updates.
removing openssl dependency (except on Linux where it's still necessary)
upgrade cpp-httplib
new cmake-presets
2026-06-15 18:51:24 -07:00
Grant Limberg ace5352388 Bump cpp-httplib so it supports win32 2026-06-15 17:41:09 -07:00
Grant Limberg deee555fbf fix deps so that architecture is taken into account 2026-06-15 17:27:00 -07:00
Grant Limberg 404fa9b2f9 fix incompatible options on crypto files in windows 2026-06-15 16:59:49 -07:00
Grant Limberg 4988d53d0e fix linking via CMake on Windows 2026-06-15 16:54:12 -07:00
Grant Limberg 14faaaa13e update to use makefiles by default on *ix 2026-06-15 16:48:58 -07:00
Grant Limberg 8ae153b44c Set thinkgs up for Windows in CMake
Also fixed a ZT_NONFREE issue
2026-06-15 16:22:17 -07:00
Grant Limberg f1c6a64035 build fix 2026-06-15 13:46:15 -07:00
Grant Limberg 0e637a1de6 More linux/mac cmake cleanup
* Brewfile for homebrew dependencies on macos
* Pull & build google-cloud-cpp/otel via bootstrap script rather than
  keeping it all in the ext/ subfolder.
* Update controller CI build with new steps
2026-06-15 12:59:32 -07:00
Grant Limberg e09e7b5c9b Fix PIC/PIE flags 2026-06-12 17:41:45 -07:00
Grant Limberg 052ea55636 bring CMake in line with make-linux/mac/bsd 2026-06-12 17:15:30 -07:00
Grant Limberg 378dd27361 main controller cmake file cleanup 2026-06-12 16:34:26 -07:00
Grant Limberg a4904c97b0 Repo cleanup.
* Remove now dead separate CV1/CV2 implementations.  Consolidated into
  CentralDB.
* Remove smeeclient.  No longer needed.
2026-06-12 16:23:04 -07:00
Grant Limberg 407bfdfd0b fix pubsub flood on ctl startup 2026-06-10 16:45:29 -07:00
Grant Limberg 85eec45372 remove db check from onlineNotificationThread 2026-06-10 15:46:12 -07:00
Grant Limberg fede9b9afb better logging of commit thread ticks
shows whats happening on each
2026-06-10 15:44:05 -07:00
Grant Limberg a3a45a75a7 Better data handling if a db write fails 2026-06-10 14:29:07 -07:00
Grant Limberg a001eba98f better closed connection handling to postgres 2026-06-10 14:05:46 -07:00
Grant Limberg af7c882315 perf(controller): drop per-row BigTable reads in status writer
writePending() did a synchronous ReadRow per queued node to avoid
rewriting unchanged os/arch/version. With thousands of check-ins per
cycle that's thousands of sequential round-trips every 10s, and the read
was nearly pointless since last_seen and the IP cell are written every
cycle anyway.

Replace it with an in-process cache of the last-written node_info per
row (keyed by a 64-bit hash of the row key). node_info is emitted only
when changed, new, or past its refresh window; check_in is still written
every cycle. Failed rows are dropped from the cache so they're rewritten
next cycle, and stale entries are evicted (24h TTL). Net result: one
BulkApply per cycle, no read RPCs.
2026-06-10 13:28:40 -07:00
Grant Limberg eec53a5866 Add handling sso networks for both CV1 & CV2 simultaneously 2026-06-02 14:54:55 -07:00
Grant Limberg 174130c080 More sso error logging & removing a few logs as well 2026-06-02 13:43:50 -07:00
Grant Limberg f76e6b916c This workflow does need to be removed from this branch.
once finally merged it'll be irrelevant.
2026-05-28 09:25:39 -07:00
Grant Limberg 3dffacc024 cargo fix 2026-05-28 09:12:38 -07:00
Grant Limberg 29f5464dbd Merge branch 'dev' into gl/ctl-pubsub 2026-05-28 08:49:48 -07:00
Grant Limberg 346dc907a1 remove noisy log 2026-05-28 08:03:07 -07:00
Grant Limberg b1427153aa handle nulls for remote trace target 2026-05-27 07:52:25 -07:00
Grant Limberg 80f5e81d8c ensure controller doesn't change assigned frontend 2026-05-19 11:09:02 -07:00
Grant Limberg 9a9a8c09c3 fix for missing field in json 2026-05-14 15:30:44 -07:00
Grant Limberg ac20c320cf update network.proto in line with CV1 and CV2 2026-05-14 13:07:13 -07:00
Grant Limberg 1c755845f0 align JSON keys between CentralDB and PubSub listener/writer
- Default enableBroadcast to true on load (initializeNetworks and _getNetwork) to match DB::initNetwork. Networks whose JSONB lacks the field now get broadcast on instead of off, which is the documented ZeroTier default.

- Stop overwriting networks_ctl.frontend on every UPDATE. It should remain whatever was set at INSERT. The early-exit guard already prevents cross-frontend writes from applying, so dropping the SET preserves correctness. Members are unaffected — they source frontend from the network row, not the change payload.

- Add enableBroadcast, creationTime, and revision to PubSubListener's network toJson. The protobuf carries them, but the listener was dropping them on inbound, which caused revision to reset on every PubSub-delivered network update.

- Rename member version keys to vMajor/vMinor/vRev/vProto in both PubSubListener (incoming) and PubSubWriter (outgoing). CentralDB, EmbeddedNetworkController, and the rest of the codebase already use these names; the listener/writer were the only sites using versionMajor/Minor/Rev/Protocol, so member version info was being silently dropped through PubSub in both directions.

- Pass newMember by const reference in ControllerChangeNotifier::notifyMemberChange to match notifyNetworkChange. Avoids a full JSON copy on every member notify.

- Remove duplicate nlohmann::json::parse call in RedisNetworkListener.
2026-05-13 11:01:39 -07:00
Grant Limberg 7b44e0a3a5 Fix json key inconsistency in pubsub writer 2026-05-13 10:40:02 -07:00
Grant Limberg b79592c71f Bugfix pass
ssoConfig checked wrong variable
DNS missing-branch wrote to input not output
SSO fields renamed to sso* and nested under ssoConfig (across CentralDB, DB defaults, migration_models)
Member identity populated into the right field; address = memberId
noAutoAssignIPs → noAutoAssignIps casing
Member version columns now SELECTed + populated (both sites)
removeTraceLevel typo in DB.cpp defaults
8. eraseMember uninitialized tmp2 — member deletes now actually delete
9. Connection leak on continue in both commit branches
10. Unsafe (uint64_t)config["revision"] cast → OSUtils::jsonInt
11. target = "NULL" literal text → std::optional<std::string> for real SQL NULL
~13 unsafe direct json→string casts → OSUtils::jsonString; dead vars removed from save(); change_source blocks use is_string() checks.
Heartbeat zadd literal "controllerId" → variable
Network revision double-increment removed
PubSub notifier now sees actual old state (DB query moved before INSERT in both branches)
_delete_network reads config["id"] instead of config["nwid"]
ip_assignments empty-string guards added at both sites (also closes G)
ensure last_modified is set on every write to networks & members
2026-05-13 10:28:15 -07:00
Grant Limberg f0522a7cdb fix a typo in some json handling that could be affecting an issue I'm tracking down 2026-05-08 10:58:05 -07:00
Grant Limberg 8c3a777108 addressing pr comments 2026-05-04 16:27:45 -07:00
Grant Limberg 23d395d781 change sso topic names for clarity 2026-04-30 09:41:24 -07:00
Grant Limberg 484d9e0d8c cargo update 2026-04-27 11:31:02 -07:00
Grant Limberg aa586efbb5 query fix 2026-04-17 13:50:49 -07:00
Grant Limberg 0e8ec661c8 make sure subscribe pulls stay running 2026-04-11 13:42:44 -07:00
Grant Limberg d2361a9a66 logging 2026-04-11 12:39:05 -07:00
Grant Limberg 910334c2a5 Fix silent PubSub message loss in controller subscription loop
The 10-second session.cancel() loop raced with in-flight acks — when
cancel fired while the GCP client was processing messages, acks were
lost before reaching the server. With message ordering enabled, an
unacked message blocks all subsequent messages on that ordering key,
causing silent stalls with no error output.

Two fixes:
- Replace the cancel/reconnect timer with a blocking session.get(),
  storing the session future so the destructor can cancel on shutdown.
- Always ack messages even when onNotification fails — permanent errors
  (bad protobuf, missing fields) will never succeed on retry and would
  otherwise poison the ordering key indefinitely.
2026-04-11 09:30:21 -07:00
Grant Limberg 4ca5c9b820 fix poison pill blocking proccessing 2026-04-11 07:40:11 -07:00
Grant Limberg 1f3a04f303 periodic queue size logging, and fix some db connection leaks 2026-04-03 10:28:28 -07:00
Grant Limberg af7eae5d9e Added a little bit more logging for the node checkin/bigtable write process 2026-04-03 09:52:55 -07:00
Grant Limberg ea5c91b0e9 Remove smee from CentralDB.
Now handled in CV1 on new member join via pubsub integration when a new member comes through
2026-04-02 10:04:23 -07:00
Grant Limberg 20f7311622 Skip redundant nonce sending with an expiry time of 0 2026-04-01 09:48:05 -07:00
Grant Limberg dd6e69f530 sso query fix in controller 2026-03-31 14:04:30 -07:00
Grant Limberg 78b25f4ae2 update settings to enable SSO networks 2026-03-31 13:24:03 -07:00
Grant Limberg 895b06033d temporary logging 2026-03-31 10:56:04 -07:00
Grant Limberg b047038ca1 print a message when the SSO PSK is configured 2026-03-31 08:48:00 -07:00
Grant Limberg 7ec42461d8 configure assigned central version in startup script 2026-03-19 11:57:35 -07:00
Grant Limberg e49b347a8f plumb through config changes for sso pubsub 2026-03-19 11:56:17 -07:00
Grant Limberg 7faf30d0cc another fix 2026-03-18 13:51:16 -07:00
Grant Limberg d9507dd895 Undo change to old migration that shouldn't have been made 2026-03-18 13:18:03 -07:00
Grant Limberg 2c57f85e25 add assigned_central_version column to controllers_ctl
Allow controllers to advertise which central version (cv1, cv2, or all)
they are assigned to handle via a new configurable field. The value is
persisted to the database on each heartbeat and validated at startup
against the DB CHECK constraint.
2026-03-17 16:30:25 -07:00
Grant Limberg a840b94e0d add SSO nonce/auth PubSub messaging with frontend routing
Publish CTL_NONCE_UPDATE to PubSub when nonces are created or reused in
getSSOAuthInfo(), with the network's frontend as a message attribute so
only the correct CV frontend receives it. Listen for ZT1_AUTH_UPDATE
messages and update sso_expiry.authentication_expiry_time accordingly,
with a network existence check before applying.

- Add sso_send_topic/sso_recv_topic to PubSubConfig
- Add PubSubWriter::publishSSONonceUpdate() with frontend param
- Add PubSubSSOListener class for inbound auth updates
- Rename CV1_AUTH_UPDATE to ZT1_AUTH_UPDATE in sso.proto
- Fix pre-existing connection pool leak in getSSOAuthInfo() catch block
2026-03-17 14:42:49 -07:00
Grant Limberg 03aa33bba7 set network member frontend based on the network its a member of 2026-03-12 09:20:05 -07:00
Grant Limberg a2340bf60c add --provenance false to avoid $IMAGE is a manifest list error 2026-03-04 11:58:43 -08:00
Grant Limberg e918ec44d5 one last fix 2026-03-04 11:47:09 -08:00
Grant Limberg 34777b6bb4 another fix for multi-arch builds 2026-03-04 11:35:56 -08:00
Grant Limberg 97db010112 fix multi-arch manifest 2026-03-04 11:12:19 -08:00
Grant Limberg e81053aed7 fix build 2026-03-04 10:57:46 -08:00
Grant Limberg 91428eacae add github action for creating CV1 controller images 2026-03-04 10:51:13 -08:00
Grant Limberg 936801a51b update gitignore 2026-03-02 14:08:07 -08:00
Grant Limberg 8f1b213994 still use use_redis column 2026-03-02 14:08:02 -08:00
Grant Limberg 7f47b2ba8f update dockerfile to get around go stdlib bug in controller build 2026-03-02 14:07:53 -08:00
Grant Limberg eac140aa73 Update use of ztc_controller table for changes in progress in CV1. 2026-03-02 13:14:16 -08:00
Grant Limberg 38f4d124b0 woops. out of order here 2026-02-25 15:04:40 -08:00
Grant Limberg 0ad6b19705 drop index 2026-02-25 14:37:27 -08:00
Grant Limberg ae7ee51b9d fix db migrations 2026-02-25 14:36:53 -08:00
Grant Limberg 0f0e6b3b16 add a linked_id column to the oidc_config table.
Required to get the list of configs  for a particular org that the controller has.  Named it `linked_id` rather than `org_id` since we don't know what it will be linked to in CV2
2026-02-25 13:37:04 -08:00
Grant Limberg c653e764b8 WIP: Update sso info retrieval method 2026-02-24 14:01:33 -08:00
Grant Limberg 35f7bf2291 disable peer metrics in Central controller 2026-01-23 13:06:02 -08:00
Grant Limberg ccb9a45d21 Remove extra verbose logging from controller 2026-01-23 11:53:06 -08:00
Grant Limberg 68a96344ed Merge branch 'dev' into gl/ctl-pusub 2026-01-23 11:18:45 -08:00
Grant Limberg 2ba50f4544 set --provenance false on docker build to try and fix docker image creation issue 2025-11-12 16:17:03 -08:00
Grant Limberg 50daf9537b controller build readme update 2025-11-11 18:40:58 -08:00
Grant Limberg 5c27068b80 Enable cross-service propagation of otel trace metadata 2025-10-30 13:47:50 +01:00
Grant Limberg 30c4484731 Set ordering key on pubsub message publishing. 2025-10-27 11:01:36 +01:00
Grant Limberg c21ff23477 Added some more logging to the ctl around deauths 2025-10-27 09:44:36 +01:00
Grant Limberg 76ba89060b ensure change source is controller if otherwise unset 2025-10-09 16:29:43 -07:00
Grant Limberg 77aa8c7bf8 missed one 2025-10-09 15:47:34 -07:00
Grant Limberg bf0fe2c09a these should be empty json arrays, not just "[]".
Actually these *should* be structs, not filling out json.  Adding this to the tech debt backlog
2025-10-09 15:39:40 -07:00
Grant Limberg 19ccc98f91 more variations on checking json 2025-10-09 15:28:48 -07:00
Grant Limberg e3f65ff359 compiler fix 2025-10-09 15:02:16 -07:00
Grant Limberg 47367673f9 more robustifying incoming messages 2025-10-09 14:47:25 -07:00
Grant Limberg 46a8cd7b67 robustify the handling of tags/capabilities/rules which are strings of encoded json 2025-10-09 14:39:46 -07:00
Grant Limberg 946d96f482 dont NACK failed messages. That gets us into a loop of failing the same messages over and over again 2025-10-09 14:34:24 -07:00
Grant Limberg 6b74bf289a fix assign mode json 2025-10-06 15:10:42 -07:00
Grant Limberg fe221b9359 debug output for IP addressing & fixing order of operations in a couple of places. Only send notification of a change to pubsub after it's been written to the DB 2025-10-06 14:24:57 -07:00
Grant Limberg 6df7366753 no need for these timeouts 2025-10-02 16:00:08 -07:00
Grant Limberg 1ef7ea0fe6 Fixing more JSON issues 2025-10-02 15:04:57 -07:00
Grant Limberg 18714c7785 add explicit nack if there's an error processing a pubsub message 2025-10-02 11:56:44 -07:00
Grant Limberg a75d06ad64 cleaning up some gross JSON code 2025-10-02 11:35:03 -07:00
Grant Limberg 4861b7df1e one last json parsing fix. but really this time 2025-10-01 17:18:31 -07:00
Grant Limberg 0d21250b62 one last json parsing fix 2025-10-01 16:33:17 -07:00
Grant Limberg 3bbf194594 pubsub writer should work now 2025-10-01 16:25:27 -07:00
Grant Limberg fb75369597 throwing stuff at the wall cuz I can't actually debug this right now 2025-10-01 16:08:06 -07:00
Grant Limberg a8b5c89efa think this is where things are crashing, but I'm not sure why yet. Comment it out to make sure 2025-10-01 15:05:34 -07:00
Grant Limberg 80f82132dd fix tags/capabilities? 2025-10-01 14:00:17 -07:00
Grant Limberg 7faf705e10 make the default an empty array here 2025-10-01 13:33:02 -07:00