On certain OSes (Linux & Apple) tun#, tap#, and of course zt# are blacklisted by default, this adds wg# to the list as WireGuard is a similar popular service with wg# being the default adapter name(s) by convention.
Unaligned access caused SIGBUS errors on ARMv6 and ARMv7 targets under FreeBSD.
This was also the cause of the repeating TAP devices. Each time the SIGBUS happened, the service would auto-restart itself, create a new TAP device, and then crash again.
The particular place causing the SIGBUS was:
https://github.com/zerotier/ZeroTierOne/blob/master/node/Utils.hpp#L695
Major new features are:
* **Multipath support** with modes modeled after the Linux kernel's bonding driver. This includes active-passive and active-active modes with fast failover and load balancing. See section 2.1.5 of the manual.
* **DNS configuration** push from network controllers to end nodes, with locally configurable permissions for whether or not push is allowed.
* **AES-GMAC-SIV** encryption mode, which is both somewhat more secure and significantly faster than the old Salsa20/12-Poly1305 mode on hardware that supports AES acceleration. This includes virtually all X86-64 chips and most ARM64. This mode is based on AES-SIV and has been audited by Trail of Bits to ensure that it is equivalent security-wise.
Known issues that are not yet fixed in this beta:
* Some Mac users have reported periods of 100% CPU in kernel_task and connection instability after leaving networks that have been joined for a period of time, or needing to kill ZeroTier and restart it to finish leaving a network. This doesn't appear to affect all users and we haven't diagnosed the root cause yet.
* The service sometimes hangs on shutdown requiring a kill -9. This also does not affect all systems or users.
* AES hardware acceleration is not yet supported on 32-bit ARM, PowerPC (32 or 64), or MIPS (32 or 64) systems. Currently supported are X86-64 and ARM64/AARCH64 with crypto extensions.
* Some users have reported multicast/broadcast outages on networks lasting up to 30 seconds. Still investigating.
We're trying to fix all these issues before the 1.6.0 release. Stay tuned.
Just adding it to the repo, but it still needs to be dealt with during install.
Probably put it in $ZT_HOME and then symlink to the proper place for the distro?
searches $ZT_HOME/networks.d/ for network IDs
searches HISTORY for 16 digit numbers that look like network IDs.
If a redis cluster member fails over to the slave, we'll get an error from not specifying the key for the insert. Recover from that instead of crashing the controller
* ARM32 platform build and flag fixes
* Add a clarification line to LICENSE.txt
* Fix license message in CLI
* Windows service now looks for service command line arguments
* Fixed a bug that could cause excessive queued multicasts
Major Changes
* Mac version no longer requires a kernel extension, instead making use of the [feth interfaces](https://apple.stackexc
* Added support for concurrent multipath (multiple paths at once) with traffic weighting by link quality and faster rec
* Added under-the-hood support for QoS (not yet exposed) that will eventually be configurable via our rules engine.
Minor Changes and Bug Fixes
* Experimental DB driver for [LF](https://github.com/zerotier/lf) to store network controller data (LFDB.cpp / LFDB.hpp
* Modified credential push and direct path push timings and algorithms to somewhat reduce "chattiness" of the protocol
* Removed our beta/half-baked integration of Central with the Windows UI. We're going to do a whole new UI of some kind
* Fixed stack overflow issues on Linux versions using musl libc.
* Fixed some alignment problems reported on ARM and ARM64, but some reports we could not reproduce so please report any
* Fixed numerous other small issues and bugs such as ARM alignment issues causing crashes on some devices.
* Windows now sets the adapter name such that it is consistent in both the Windows UI and command line utilities.
Because zerotier-one depends on "one", which is a phony target,
making it relinks zerotier-one every time, which is probably
not the desired behavior.
In any case, zerotier-one, zerotier-cli, and zerotier-idtool are
real target, so they should have real dependencies and actions.
The "one" target now depends on these three files.
Behavior is mostly unchanged otherwise.
Made the smallest change I could figure out.
Had to remove -Werror because something in the postgres adapter woudn't
compile.
brew install postgresql rabbitmq-c
and seems to put the libs in the right place
This commit changes the Dockerfile ot use the multi-stage build and help
to get an automated build on Docker.
The idea of the multi-stage build is to use the already stable Debian
distribution channel to provide up-to-date versions of ZeroTier.
The benefit is that it would be possible to automate the image build,
either on Docker Hub, Travis or taking advantage of the [docker-library/official-images] infrastructure.
This changes follows the best-practices suggested by
[docker-library/official-images], such as using a High Availability GPG
keyserver, providing a default CMD, allowing "bash" on `docker run` and
others.
Given that both the builder `debian:stretch` and `alpine:latest` are
official messages and have [manifests], this means that this Dockerfile
is also multi-platform. This means that this same Dockerfile will
pick-up the correct Debian package according to the architecture of the
running system during build.
With this changes we could try to promote the image to be parte of
[docker-library/official-images], and take advantage of automated
publishing of multi-architecture images. Others would be able to use
`docker run zerotier` and download the latest version appropriate to
their system.
Related to #682
[docker-library/official-images]: https://github.com/docker-library/official-images
[manifests]: https://blog.docker.com/2017/09/docker-official-images-now-multi-platform/
1) Use existing standard libraries to convert to/from IPv4/IPv6 strings and binary representation.
2) Move null terminator assignment InetAddress::toIpString to top of function, this way if ANY errors occurs that don't write content to the buffer, we're not passing a potentially dangerous buffer around.
* Fixed a bug that caused exits to take a long time on Mac due to huge numbers of redundant attempts to delete managed routes.
* Fixed a socket limit problem on Windows that caused the ZeroTier service to run out of sockets, causing the UI and CLI to be unable to access the API.
* Fixed a threading bug in the ZeroTier Core, albeit one that never manifested on the regular ZeroTier One service/client.
* Fixed a bug that could cause the service to crash if an authorized local client accessed an invalid URL via the control API. (Not exploitable since you needed admin access anyway.)
Added LinuxNetLink to talk to the rtnetlink socket for adding interfaces, addresses routes. Not yet complete. Can currently monitor changes on the system.
when building with `ZT_DEBUG=1` this hint produces a warning:
> node/Packet.cpp:335:43: error: 'register' storage class specifier is deprecated and incompatible with C++17 [-Werror,-Wdeprecated-register]
See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4340
The Hyperledger implementation (https://github.com/hyperledger/iroha-ed25519)
contains changes to the assembly code to allow PIC. This in turn fixes
compilation/linking of ZeroTier One when "full hardening" flags are used.
* Features and Core Improvements
* Path selection has been overhauled to improve path stability, simplify code, and prepare for multi-path and trunking in the next major release.
* This version introduces remote tracing for remote diagnostics. Network controllers can set a node (usually the controller itself) to receive remote tracing events from all members of the network or from select members. Events are only sent if they pertain to a given network for security reasons.
* Multicast replication can now be done by designated multicast replicators on a network (flagged as such at the controller) rather than by the sender. Most users won't want this, but it's useful for specialized use cases on hub-and-spoke networks and for low-power devices.
* Cryptographic performance improvements on several platforms.
* Multithreaded performance improvements throughout the code base, including the use of an inline lightweight spinlock for low-contention resources.
* Bugs fixed
* Disappearing routes on Mac (GitHub issue #600)
* Route flapping and path instability in some dual-stack V4/V6 networks
* Blacklist (in local.conf) doesn't work reliably (GitHub issue #656)
* Connection instabilities due to unsigned integer overflows in timing comparisons (use int64_t instead of uint64_t)
* Binaries don't run on some older or lower-end 32-bit ARM chips (build problem)
* ARM NEON crypto code crashes (build problem)
* Fixed some lock ordering issues revealed by "valgrind" tool
* The "zerotier-idtool" command could not be accessed from "zerotier-one" via command line switch
* Leaking sockets on some platforms when uPnP/NAT-PMP is enabled
* Fixed two very rare multithreading issues that were only observed on certain systems
* Platform-Specific Changes
* MacOS
* Installer now loads the kernel extension right away so that High Sierra users will see the prompt to authorize it. This is done in the "Security & Privacy" preference pane and must be done driectly on the console (not via remote desktop). On High Sierra and newer kexts must be authorized at the console via security settings system preferences pane.
* Windows
* The Windows installer should now install the driver without requiring a special prompt in most cases. This should make it easier for our packages to be accepted into and updated in the Chocolatey repository and should make it easier to perform remote installs across groups of machines using IT management and provisioning tools.
* The Windows official packages are now signed with an EV certificate (with hardware key).
* The Windows UI can now log into ZeroTier Central and join networks via the Central API.
* The `zerotier-idtool` command should now work on Windows without ugly hacks.
* Upgraded the installer version.
* Made a few changes to hopefully fix sporadic "will not uninstall" problems, though we cannot duplicate these issues ourselves.
* Linux
* Device names are now generated deterministically based on network IDs for all newly joined networks.
* Android
* Multicast now works on Android in most cases! Android apps can send and receive multicast and subscribe to multicast group IPs. Note that in some cases the app must bind to the specific correct interface for this to work.
* IPv6 can be disabled in UI for cases where it causes problems.
* Path selection has been overhauled to improve path stability, simplify code, and prepare for multi-path and trunking in the next major release.
* This version introduces remote tracing for remote diagnostics. Network controllers can set a node (usually the controller itself) to receive remote tracing events from all members of the network or from select members. Events are only sent if they pertain to a given network for security reasons.
* Multicast replication can now be done by designated multicast replicators on a network (flagged as such at the controller) rather than by the sender. Most users won't want this, but it's useful for specialized use cases on hub-and-spoke networks and for low-power devices.
* Cryptographic performance improvements on several platforms.
* Multithreaded performance improvements throughout the code base, including the use of an inline lightweight spinlock for low-contention resources.
* Bugs fixed
* Disappearing routes on Mac (GitHub issue #600)
* Route flapping and path instability in some dual-stack V4/V6 networks
* Blacklist (in local.conf) doesn't work reliably (GitHub issue #656)
* Connection instabilities due to unsigned integer overflows in timing comparisons (use int64_t instead of uint64_t)
* Binaries don't run on some older or lower-end 32-bit ARM chips (build problem)
* ARM NEON crypto code crashes (build problem)
* Fixed some lock ordering issues revealed by "valgrind" tool
* The "zerotier-idtool" command could not be accessed from "zerotier-one" via command line switch
* Leaking sockets on some platforms when uPnP/NAT-PMP is enabled
* Fixed two very rare multithreading issues that were only observed on certain systems
* Platform-Specific Changes
* MacOS
* Installer now loads the kernel extension right away so that High Sierra users will see the prompt to authorize it. This is done in the "Security & Privacy" preference pane and must be done driectly on the console (not via remote desktop). On High Sierra and newer kexts must be authorized at the console via security settings system preferences pane.
* Windows
* The Windows installer should now install the driver without requiring a special prompt in most cases. This should make it easier for our packages to be accepted into and updated in the Chocolatey repository and should make it easier to perform remote installs across groups of machines using IT management and provisioning tools.
* The Windows official packages are now signed with an EV certificate (with hardware key).
* The Windows UI can now log into ZeroTier Central and join networks via the Central API.
* The `zerotier-idtool` command should now work on Windows without ugly hacks.
* Upgraded the installer version.
* Made a few changes to hopefully fix sporadic "will not uninstall" problems, though we cannot duplicate these issues ourselves.
* Linux
* Device names are now generated deterministically based on network IDs for all newly joined networks.
* Android
* Multicast now works on Android in most cases! Android apps can send and receive multicast and subscribe to multicast group IPs. Note that in some cases the app must bind to the specific correct interface for this to work.
* IPv6 can be disabled in UI for cases where it causes problems.
Adds a "vault" section to local.conf. Example local.conf:
{
"config": {
"vault": {
"vaultURL": "https://some.vault.host:8200",
"vaultToken": "my-super-secret-vault-token",
"vaultPath": "secure/place/to/put/identity"
}
}
Additionally, the following environment variables can be set. Environment variables override local.conf:
VAULT_ADDR
VAULT_TOKEN
VAULT_PATH
Identities will be placed in the keys "public" and "secret" under the user specified path. If no path is specified, they will be placed in the token specific cubbyhole.
If identity.public and identity.secret exist on disk and vault is configured, they will be automatically added to Vault and removed from disk.
TODO:
* Decide behavior for if Vault cannot be reached.
* Add libcurl as a dependency in Linux & Mac builds
* Add libcurl as a requirement for linux packages
When running ArchlinuxARM with 64bit support on Raspberry Pi3, the default build fails due to the architecture look-up at the beginning of the build process. A simple addition of the Armv7l section above allows the build to continue and successfully run.
There were cases in the code where time calculations and comparisons were overflowing and causing connection instability. This will keep time calculations within expected ranges.
I've only tested locally, on a power8 box running Ubuntu 16.10, but everything (and earth) checks out.
Let me know if more testing infrastructure is needed.
* Managed routes are now only bifurcated for the default route. This is a change in behavior, though few people will probably notice. Bifurcating all managed routes was causing more trouble than it was worth for most users.
* Up to 2X crypto speedup on x86-64 (except Windows, which will take some porting) and 32-bit ARM platforms due to integration of fast assembly language implementations of Salsa20/12 from the [supercop](http://bench.cr.yp.to/supercop.html) code base. These were written by Daniel J. Bernstein and are in the public domain. My Macbook Pro (Core i5 2.8ghz) now does almost 1.5GiB/sec Salsa20/12 per core and a Raspberry Pi got a 2X boost. 64-bit ARM support and Windows support will take some work but should not be too hard.
* Refactored code that manages credentials to greatly reduce memory use in most cases. This may also result in a small performance improvement.
* Reworked and simplified path selection and priority logic to fix path instability and dead path persistence edge cases. There have been some sporadic reports of persistent path instabilities and dead paths hanging around that take minutes to resolve. These have proven difficult to reproduce in house, but hopefully this will fix them. In any case it seems to speed up path establishment in our tests and it makes the code simpler and more readable.
* Eliminated some unused cruft from the code around path management and in the peer class.
* Fixed an issue causing build problems on some MIPS architecture systems.
* Fixed Windows forgetting routes on sleep/wake or in some other circumstances. (GitHub issue #465)
Version 1.2.2 fixes a few bugs discovered after the 1.2.0 release. These are:
* A bug causing unreliable multicast propagation (GitHub issue #461).
* A crash in ARM binaries due to a build chain and flags problem.
* A bug in the network controller preventing members from being listed (GitHub issue #460).
If there is something you'd like to have added to ZeroTier, to go to https://discuss.zerotier.com/c/feature-requests/ instead. Issues there can be voted on and discussed in-depth.
about:Game issues are better served by forum posts
title:Please go to our Discuss or Reddit for game-related issues. Thanks!
labels:wontfix
assignees:''
---
Are you having trouble connecting to a game on your virtual network after installing ZeroTier?
- [ ] Yes
- [ ] No
If you answered yes, then it is very likely that your question would be better answered on our [Community Forums](https://discuss.zerotier.com) or [Reddit](https://www.reddit.com/r/zerotier/) community; we monitor both regularly. We also have extensive documentation on our [Knowledge Base](https://zerotier.atlassian.net/wiki/spaces/SD/overview). Thank you!
# Authors and Third Party Code Licensing Information
## Primary Authors
* ZeroTier Core and ZeroTier One virtual networking service<br>
Adam Ierymenko / adam.ierymenko@zerotier.com
Joseph Henry / joseph.henry@zerotier.com (QoS and multipath)
* Java JNI Interface to enable Android application development, and Android app itself (code for that is elsewhere)<br>
Grant Limberg / glimberg@gmail.com
@@ -25,7 +28,7 @@
## Third-Party Code
ZeroTier includes the following third party code, either in ext/ or incorporated into the ZeroTier core.
ZeroTier includes the following third party code, either in ext/ or incorporated into the ZeroTier core. This third party code remains licensed under its original license and is not subject to ZeroTier's BSL license.
* LZ4 compression algorithm by Yann Collet
@@ -45,13 +48,6 @@ ZeroTier includes the following third party code, either in ext/ or incorporated
* Home page: https://github.com/nlohmann/json
* License grant: MIT
* TunTapOSX by Mattias Nissler
* Files: ext/tap-mac/tuntap/*
* Home page: http://tuntaposx.sourceforge.net/
* License grant: BSD attribution no-endorsement
* ZeroTier Modifications: change interface name to zt#, increase max MTU, increase max devices
* tap-windows6 by the OpenVPN project
* Files: windows/TapDriver6/*
@@ -71,3 +67,9 @@ ZeroTier includes the following third party code, either in ext/ or incorporated
* Files: ext/libnatpmp/* ext/miniupnpc/*
* Home page: http://miniupnp.free.fr/
* License grant: BSD attribution no-endorsement
* cpp-httplib by yhirose
* Files: ext/cpp-httplib/*
* Home page: https://github.com/yhirose/cpp-httplib
The final .AIP file can only be edited on Windows with [Advanced Installer Enterprise](http://www.advancedinstaller.com/). In addition to incrementing the version be sure that a new product code is generated. (The "upgrade code" GUID on the other hand must never change.)
@@ -28,36 +28,6 @@ Mac's easy. Just type:
You will need [Packages](http://s.sudre.free.fr/Software/Packages/about.html) and our release signing key in the keychain.
## Linux
Mount the GPG key for *contact@zerotier.com* and then on an x86_64 box with a recent version of Docker and an Internet connection run:
make distclean
cd linux-build-farm
./build.sh
This will build i386 and x86_64 packages. Now ssh into our build Raspberry Pi and type `make debian` there to build the Raspbian armhf package. Copy it to `debian-jessie/` inside `linux-build-farm` so that it will be included in the repositories we generate. Now generate the YUM and APT repos:
rm -rf ~/.aptly*
rm -rf /tmp/zt-rpm-repo
./make-apt-repos.sh
./make-rpm-repos.sh
This will require the passphrase for *contact@zerotier.com*.
The contents of ~/.aptly/public must be published as `debian/` on `download.zerotier.com`. The contents of /tmp/zt-rpm-repo are published as `redhat/` on same.
## Windows
First load the Visual Studio solution and rebuild the UI and ZeroTier One in both x64 and i386 `Release` mode. Then load [Advanced Installer Enterprise](http://www.advancedinstaller.com/), check that the version is correct, and build. The build will fail if any build artifacts are missing, and Windows must have our product singing key (from DigiCert) available to sign the resulting MSI file. The MSI must then be tested on at least a few different CLEAN Windows VMs to ensure that the installer is valid and properly signed.
*After the MSI is published to download.zerotier.com in the proper RELEASE/#.#.#/dist subfolder for its version* the Chocolatey package must be rebuilt and published. Open a command prompt, change to `ext/installfiles/windows/chocolatey`, and type `choco pack`. Then use `choco push` to push it to Chocolatey (API key required).
Note that this does not cover rebuilding the drivers or their containing MSI projects, as this is typically not necessary and they are shipped in binary form in the repository for convenience.
## iOS, Android
... no docs here yet since this is done entirely out of band with regular installs.
ZeroTier is an enterprise Ethernet switch for planet Earth.
ZeroTier is a smart programmable Ethernet switch for planet Earth. It allows all networked devices, VMs, containers, and applications to communicate as if they all reside in the same physical data center or cloud region.
It erases the LAN/WAN distinction and makes VPNs, tunnels, proxies, and other kludges arising from the inflexible nature of physical networks obsolete. Everything is encrypted end-to-end and traffic takes the most direct (peer to peer) path available.
This is accomplished by combining a cryptographically addressed and secure peer to peer network (termed VL1) with an Ethernet emulation layer somewhat similar to VXLAN (termed VL2). Our VL2 Ethernet virtualization layer includes advanced enterprise SDN features like fine grained access control rules for network micro-segmentation and security monitoring.
Visit [ZeroTier's site](https://www.zerotier.com/) for more information and [pre-built binary packages](https://www.zerotier.com/download.shtml). Apps for Android and iOS are available for free in the Google Play and Apple app stores.
All ZeroTier traffic is encrypted end-to-end using secret keys that only you control. Most traffic flows peer to peer, though we offer free (but slow) relaying for users who cannot establish peer to peer connections.
The goals and design principles of ZeroTier are inspired by among other things the original [Google BeyondCorp](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43231.pdf) paper and the [Jericho Forum](https://en.wikipedia.org/wiki/Jericho_Forum) with its notion of "deperimeterization."
Visit [ZeroTier's site](https://www.zerotier.com/) for more information and [pre-built binary packages](https://www.zerotier.com/download/). Apps for Android and iOS are available for free in the Google Play and Apple app stores.
ZeroTier is licensed under the [BSL version 1.1](https://mariadb.com/bsl11/). See [LICENSE.txt](LICENSE.txt) and the [ZeroTier pricing page](https://www.zerotier.com/pricing) for details. ZeroTier is free to use internally in businesses and academic institutions and for non-commercial purposes. Certain types of commercial use such as building closed-source apps and devices based on ZeroTier or offering ZeroTier network controllers and network management as a SaaS service require a commercial license.
A small amount of third party code is also included in ZeroTier and is not subject to our BSL license. See [AUTHORS.md](AUTHORS.md) for a list of third party code, where it is included, and the licenses that apply to it. All of the third party code in ZeroTier is liberally licensed (MIT, BSD, Apache, public domain, etc.).
### Getting Started
ZeroTier's basic operation is easy to understand. Devices have 10-digit *ZeroTier addresses*like `89e92ceee5` and networks have 16-digit network IDs like `8056c2e21c000001`. All it takes for a device to join a network is its 16-digit ID, and all it takes for a network to authorize a device is its 10-digit address. Everything else is automatic.
Everything in the ZeroTier world is controlled by two types of identifier: 40-bit/10-digit *ZeroTier addresses*and 64-bit/16-digit *network IDs*. These identifiers are easily distinguished by their length. A ZeroTier address identifies a node or "device" (laptop, phone, server, VM, app, etc.) while a network ID identifies a virtual Ethernet network that can be joined by devices.
A "device" in our terminology is any "unit of compute" capable of talking to a network: desktops, laptops, phones, servers, VMs/VPSes, containers, and even user-space applications via our [SDK](https://github.com/zerotier/ZeroTierSDK).
ZeroTier addresses can be thought of as port numbers on an enormous planet-wide enterprise Ethernet smart switch supporting VLANs. Network IDs are VLAN IDs to which these ports may be assigned. A single port can be assigned to more than one VLAN.
For testing purposes we provide a public virtual network called *Earth* with network ID `8056c2e21c000001`. You can join it with:
sudo zerotier-cli join 8056c2e21c000001
Now wait about 30 seconds and check your system with `ip addr list` or `ifconfig`. You'll see a new interface whose name starts with *zt* and it should quickly get an IPv4 and an IPv6 address. Once you see it get an IP, try pinging `earth.zerotier.net` at `29.209.112.93`. If you've joined Earth from more than one system, try pinging your other machine. If you don't want to belong to a giant Ethernet party line anymore, just type:
sudo zerotier-cli leave 8056c2e21c000001
The *zt* interface will disappear. You're no longer on the network.
To create networks of your own, you'll need a network controller. ZeroTier One (for desktops and servers) includes controller functionality in its default build that can be configured via its JSON API (see [README.md in controller/](controller/)). ZeroTier provides a hosted solution with a nice web UI and SaaS add-ons at [my.zerotier.com](https://my.zerotier.com/). Basic controller functionality is free for up to 100 devices.
A ZeroTier address looks like `8056c2e21c` and a network ID looks like `8056c2e21c000001`. Network IDs are composed of the ZeroTier address of that network's primary controller and an arbitrary 24-bit ID that identifies the network on this controller. Network controllers are roughly analogous to SDN controllers in SDN protocols like [OpenFlow](https://en.wikipedia.org/wiki/OpenFlow), though as with the analogy between VXLAN and VL2 this should not be read to imply that the protocols or design are the same. You can use our convenient and inexpensive SaaS hosted controllers at [my.zerotier.com](https://my.zerotier.com/) or [run your own controller](controller/) if you don't mind messing around with JSON configuration files or writing scripts to do so.
### Project Layout
The base path contains the ZeroTier One service main entry point (`one.cpp`), self test code, makefiles, etc.
-`artwork/`: icons, logos, etc.
-`attic/`: old stuff and experimental code that we want to keep around for reference.
-`controller/`: the reference network controller implementation, which is built and included by default on desktop and server build targets.
@@ -38,42 +38,33 @@ To create networks of your own, you'll need a network controller. ZeroTier One (
-`macui/`: a Macintosh menu-bar app for controlling ZeroTier One, written in Objective C.
-`node/`: the ZeroTier virtual Ethernet switch core, which is designed to be entirely separate from the rest of the code and able to be built as a stand-alone OS-independent library. Note to developers: do not use C++11 features in here, since we want this to build on old embedded platforms that lack C++11 support. C++11 can be used elsewhere.
-`osdep/`: code to support and integrate with OSes, including platform-specific stuff only built for certain targets.
-`rule-compiler/`: JavaScript rules language compiler for defining network-level rules.
-`service/`: the ZeroTier One service, which wraps the ZeroTier core and provides VPN-like connectivity to virtual networks for desktops, laptops, servers, VMs, and containers.
-`tcp-proxy/`: TCP proxy code run by ZeroTier, Inc. to provide TCP fallback (this will die soon!).
-`windows/`: Visual Studio solution files, Windows service code for ZeroTier One, and the Windows task bar app UI.
The base path contains the ZeroTier One service main entry point (`one.cpp`), self test code, makefiles, etc.
-`windows/`: Visual Studio solution files, Windows service code, and the Windows task bar app UI.
### Build and Platform Notes
To build on Mac and Linux just type `make`. On FreeBSD and OpenBSD `gmake` (GNU make) is required and can be installed from packages or ports. For Windows there is a Visual Studio solution in `windows/'.
To build on Mac and Linux just type `make`. On FreeBSD and OpenBSD `gmake` (GNU make) is required and can be installed from packages or ports. For Windows there is a Visual Studio solution in `windows/`.
- **Mac**
- Xcode command line tools for OSX 10.7 or newer are required.
- Tap device driver kext source is in `ext/tap-mac` and a signed pre-built binary can be found in `ext/bin/tap-mac`. You should not need to build it yourself. It's a fork of [tuntaposx](http://tuntaposx.sourceforge.net) with device names changed to `zt#`, support for a larger MTU, and tun functionality removed.
- Xcode command line tools for OSX 10.8 or newer are required.
- **Linux**
- The minimum compiler versions required are GCC/G++ 4.9.3 or CLANG/CLANG++ 3.4.2.
- The minimum compiler versions required are GCC/G++ 4.9.3 or CLANG/CLANG++ 3.4.2. (Install `clang` on CentOS 7 as G++ is too old.)
- Linux makefiles automatically detect and prefer clang/clang++ if present as it produces smaller and slightly faster binaries in most cases. You can override by supplying CC and CXX variables on the make command line.
- CentOS 7 ships with a version of GCC/G++ that is too old, but a new enough version of CLANG can be found in the *epel* repositories. Type `yum install epel-release` and then `yum install clang` to build there.
- **Windows**
- Windows 7 or newer (and equivalent server versions) are supported. This *may* work on Vista but you're on your own there. Windows XP is not supported since it lacks many important network API functions.
- We build with Visual Studio 2015. Older versions may not work with the solution file and project files we ship and may not have new enough C++11 support.
- Pre-built signed Windows drivers are included in `ext/bin/tap-windows-ndis6`. The MSI files found there will install them on 32-bit and 64-bit systems. (These are included in our multi-architecture installer as chained MSIs.)
- Windows builds are more painful in general than other platforms and are for the adventurous.
- Windows 7 or newer is supported. This *may* work on Vista but isn't officially supported there. It will not work on Windows XP.
- We build with Visual Studio 2017. Older versions may not work. Clang or MinGW will also probably work but may require some makefile hacking.
- **FreeBSD**
- Tested most recently on FreeBSD-11. Older versions may work but we're not sure.
- GCC/G++ 4.9 and gmake are required. These can be installed from packages or ports. Type `gmake` to build.
- GNU make is required. Type `gmake` to build.
- **OpenBSD**
- There is a limit of four network memberships on OpenBSD as there are only four tap devices (`/dev/tap0` through `/dev/tap3`). We're not sure if this can be increased.
- OpenBSD lacks `getifmaddrs` (or any equivalent method) to get interface multicast memberships. As a result multicast will only work on OpenBSD for ARP and NDP (IP/MAC lookup) and not for other purposes.
- Only tested on OpenBSD 6.0. Older versions may not work.
- GCC/G++ 4.9 and gmake are required and can be installed using `pkg_add` or from ports. They get installed in `/usr/local/bin` as `egcc` and `eg++` and our makefile is pre-configured to use them on OpenBSD.
- There is a limit of four network memberships on OpenBSD as there are only four tap devices (`/dev/tap0` through `/dev/tap3`).
- GNU make is required. Type `gmake` to build.
Typing `make selftest` will build a *zerotier-selftest* binary which unit tests various internals and reports on a few aspects of the build environment. It's a good idea to try this on novel platforms or architectures.
### Running
Running *zerotier-one* with -h will show help.
Running *zerotier-one* with `-h` option will show help.
On Linux and BSD you can start the service with:
@@ -81,7 +72,7 @@ On Linux and BSD you can start the service with:
A home folder for your system will automatically be created.
The service is controlled via the JSON API, which by default is available at 127.0.0.1 port 9993. We include a *zerotier-cli* command line utility to make API calls for standard things like joining and leaving networks. The *authtoken.secret* file in the home folder contains the secret token for accessing this API. See README.md in [service/](service/) for API documentation.
The service is controlled via the JSON API, which by default is available at 127.0.0.1 port 9993. We include a *zerotier-cli* command line utility to make API calls for standard things like joining and leaving networks. The *authtoken.secret* file in the home folder contains the secret token for accessing this API. See [service/README.md](service/README.md) for API documentation.
Here's where home folders live (by default) on each OS:
@@ -96,7 +87,7 @@ Running ZeroTier One on a Mac is the same, but OSX requires a kernel extension.
This will create the home folder for Mac, place *tap.kext* there, and set its modes correctly to enable ZeroTier One to manage it with *kextload* and *kextunload*.
### Troubleshooting
### Basic Troubleshooting
For most users, it just works.
@@ -112,14 +103,6 @@ ZeroTier One peers will automatically locate each other and communicate directly
Users behind certain types of firewalls and "symmetric" NAT devices may not able able to connect to external peers directly at all. ZeroTier has limited support for port prediction and will *attempt* to traverse symmetric NATs, but this doesn't always work. If P2P connectivity fails you'll be bouncing UDP packets off our relay servers resulting in slower performance. Some NAT router(s) have a configurable NAT mode, and setting this to "full cone" will eliminate this problem. If you do this you may also see a magical improvement for things like VoIP phones, Skype, BitTorrent, WebRTC, certain games, etc., since all of these use NAT traversal techniques similar to ours.
If you're interested, there's a [technical deep dive about NAT traversal on our blog](https://www.zerotier.com/blog/?p=226). A troubleshooting tool to help you diagnose NAT issues is planned for the future as are uPnP/IGD/NAT-PMP and IPv6 transport.
If a firewall between you and the Internet blocks ZeroTier's UDP traffic, you will fall back to last-resort TCP tunneling to rootservers over port 443 (https impersonation). This will work almost anywhere but is *very slow* compared to UDP or direct peer to peer connectivity.
### Contributing
Please make pull requests against the `dev` branch. The `master` branch is release, and `edge` is for unstable and work in progress changes and is not likely to work.
### License
The ZeroTier source code is open source and is licensed under the GNU GPL v3 (not LGPL). If you'd like to embed it in a closed-source commercial product or appliance, please e-mail [contact@zerotier.com](mailto:contact@zerotier.com) to discuss commercial licensing. Otherwise it can be used for free.
Additional help can be found in our [knowledge base](https://zerotier.atlassian.net/wiki/spaces/SD/overview).
* Fix a bug in potential network path filtering that could in some circumstances lead to "software laser" effects.
* Fix a printf overflow in zerotier-cli (not exploitable or a security risk)
* Windows now looks up the name of ZeroTier devices instead of relying on them having "ZeroTier" in them.
# 2021-02-15 -- Version 1.6.4
* The groundhog saw his shadow, which meant that the "connection coma" bug still wasn't gone. We think we found it this time.
# 2021-02-02 -- Version 1.6.3
* Likely fix for GitHub issue #1334, an issue that could cause ZeroTier to
go into a "coma" on some networks.
* Also groundhog day
# 2020-11-30 -- Version 1.6.2
* Fix an ARM hardware AES crypto issue (not an exploitable vulnerability).
* Fix a Linux network leave hang due to a mutex deadlock.
# 2020-11-24 -- Version 1.6.1
This release fixes some minor bugs and other issues in 1.6.0.
* Fixed a bug that caused IP addresses in the 203.0.0.0/8 block to be miscategorized as not being in global scope.
* Changed Linux builds to (hopefully) fix LXC and SELinux issues.
* Fixed unaligned memory access that caused crash on FreeBSD systems on the ARM architecture.
* Merged CLI options for controlling bonded devices into the beta multipath code.
* Updated Windows driver with Microsoft cross-signing to fix issues on some Windows systems.
# 2020-11-19 -- Version 1.6.0
Version 1.6.0 is a major release that incorporates back-ported features from the 2.0 branch, which is still under development. It also fixes a number of issues.
New features and improvements (including those listed under 1.5.0):
* **Apple Silicon** (MacOS ARM64) native support via universal binary. ZeroTier now requires the very latest Xcode to build.
* **Linux performance improvements** for up to 25% faster tun/tap I/O performance on multi-core systems.
* **Multipath support** with modes modeled after the Linux kernel's bonding driver. This includes active-passive and active-active modes with fast failover and load balancing. See section 2.1.5 of the manual.
* **DNS configuration** push from network controllers to end nodes, with locally configurable permissions for whether or not push is allowed.
* **AES-GMAC-SIV** encryption mode, which is both somewhat more secure and significantly faster than the old Salsa20/12-Poly1305 mode on hardware that supports AES acceleration. This includes virtually all X86-64 chips and most ARM64. This mode is based on AES-SIV and has been audited by Trail of Bits to ensure that it is equivalent security-wise.
Bug fixes:
* **Managed route assignment fixes** to eliminate missing routes on Linux and what we believe to be the source of sporadic high CPU usage on MacOS.
* **Hang on shutdown** issues should be fixed.
* **Sporadic multicast outages** should be fixed.
Known remaining issues:
* AES hardware acceleration is not yet supported on 32-bit ARM, PowerPC (32 or 64), or MIPS (32 or 64) systems. Currently supported are X86-64 and ARM64/AARCH64 with crypto extensions.
# 2020-10-05 -- Version 1.5.0 (actually 1.6.0-beta1)
Version 1.6.0 (1.5.0 is a beta!) is a significant release that incorporates a number of back-ported fixes and features from the ZeroTier 2.0 tree.
Major new features are:
* **Multipath support** with modes modeled after the Linux kernel's bonding driver. This includes active-passive and active-active modes with fast failover and load balancing. See section 2.1.5 of the manual.
* **DNS configuration** push from network controllers to end nodes, with locally configurable permissions for whether or not push is allowed.
* **AES-GMAC-SIV** encryption mode, which is both somewhat more secure and significantly faster than the old Salsa20/12-Poly1305 mode on hardware that supports AES acceleration. This includes virtually all X86-64 chips and most ARM64. This mode is based on AES-SIV and has been audited by Trail of Bits to ensure that it is equivalent security-wise.
Known issues that are not yet fixed in this beta:
* Some Mac users have reported periods of 100% CPU in kernel_task and connection instability after leaving networks that have been joined for a period of time, or needing to kill ZeroTier and restart it to finish leaving a network. This doesn't appear to affect all users and we haven't diagnosed the root cause yet.
* The service sometimes hangs on shutdown requiring a kill -9. This also does not affect all systems or users.
* AES hardware acceleration is not yet supported on 32-bit ARM, PowerPC (32 or 64), or MIPS (32 or 64) systems. Currently supported are X86-64 and ARM64/AARCH64 with crypto extensions.
* Some users have reported multicast/broadcast outages on networks lasting up to 30 seconds. Still investigating.
We're trying to fix all these issues before the 1.6.0 release. Stay tuned.
# 2019-08-30 -- Version 1.4.6
* Update default root list to latest
* ARM32 platform build and flag fixes
* Add a clarification line to LICENSE.txt
* Fix license message in CLI
* Windows service now looks for service command line arguments
* Fixed a bug that could cause excessive queued multicasts
# 2019-08-23 -- Version 1.4.4
* Change license from GPL3 to BSL 1.1, see LICENSE.txt
* Fix an issue with the "ipauth" rule and auto-generated unforgeable IPv6 addresses
* Fix socket/bind errors setting IPs and routes on Linux
# 2019-08-12 -- Version 1.4.2
* Fix high CPU use bug on some platforms
* Fix issues with PostgreSQL controller DB (only affects Central)
* Restore backward compatibility with MacOS versions prior to 10.13
# 2019-07-29 -- Version 1.4.0
### Major Changes
* Mac version no longer requires a kernel extension, instead making use of the [feth interfaces](https://apple.stackexchange.com/questions/337715/fake-ethernet-interfaces-feth-if-fake-anyone-ever-seen-this).
* Added support for concurrent multipath (multiple paths at once) with traffic weighting by link quality and faster recovery from lost links.
* Added under-the-hood support for QoS (not yet exposed) that will eventually be configurable via our rules engine.
### Minor Changes and Bug Fixes
* Experimental controller DB driver for [LF](https://github.com/zerotier/lf) to store network controller data (LFDB.cpp / LFDB.hpp).
* Modified credential push and direct path push timings and algorithms to somewhat reduce "chattiness" of the protocol when idle. More radical background overhead reductions will have to wait for the 2.x line.
* Removed our beta/half-baked integration of Central with the Windows UI. We're going to do a whole new UI of some kind in the future at least for Windows and Mac.
* Fixed stack overflow issues on Linux versions using musl libc.
* Fixed some alignment problems reported on ARM and ARM64, but some reports we could not reproduce so please report any issues with exact chip, OS/distro, and ZeroTier version in use.
* Fixed numerous other small issues and bugs such as ARM alignment issues causing crashes on some devices.
* Windows now sets the adapter name such that it is consistent in both the Windows UI and command line utilities.
# 2018-07-27 -- Version 1.2.12
* Fixed a bug that caused exits to take a long time on Mac due to huge numbers of redundant attempts to delete managed routes.
* Fixed a socket limit problem on Windows that caused the ZeroTier service to run out of sockets, causing the UI and CLI to be unable to access the API.
* Fixed a threading bug in the ZeroTier Core, albeit one that never manifested on the regular ZeroTier One service/client.
* Fixed a bug that could cause the service to crash if an authorized local client accessed an invalid URL via the control API. (Not exploitable since you needed admin access anyway.)
# 2018-05-08 -- Version 1.2.10
* Fix bug loading `moons.d/` files for federated root operation.
* Fix compile problem with ZT_DEBUG on some versions of `clang`
* Fix slow network startup bug related to loading of `networks.d/` cache files
# 2018-04-27 -- Version 1.2.8
* Linux version once again builds with PIE (position independent executable) flags
* Fixed bug in zerotier-idtool file sign and verify
* Fixed minor OSX app typo
* Merged alpha NetBSD support (mostly untested, so YMMV)
* Merged several minor typo and one-liner bug fixes
# 2018-04-17 -- Version 1.2.6
* Features and Core Improvements
* Path selection has been overhauled to improve path stability, simplify code, and prepare for multi-path and trunking in the next major release.
* This version introduces remote tracing for remote diagnostics. Network controllers can set a node (usually the controller itself) to receive remote tracing events from all members of the network or from select members. Events are only sent if they pertain to a given network for security reasons.
* Multicast replication can now be done by designated multicast replicators on a network (flagged as such at the controller) rather than by the sender. Most users won't want this, but it's useful for specialized use cases on hub-and-spoke networks and for low-power devices.
* Cryptographic performance improvements on several platforms.
* Multithreaded performance improvements throughout the code base, including the use of an inline lightweight spinlock for low-contention resources.
* Bugs fixed
* Disappearing routes on Mac (GitHub issue #600)
* Route flapping and path instability in some dual-stack V4/V6 networks
* Blacklist (in local.conf) doesn't work reliably (GitHub issue #656)
* Connection instabilities due to unsigned integer overflows in timing comparisons (use int64_t instead of uint64_t)
* Binaries don't run on some older or lower-end 32-bit ARM chips (build problem)
* ARM NEON crypto code crashes (build problem)
* Fixed some lock ordering issues revealed by "valgrind" tool
* The "zerotier-idtool" command could not be accessed from "zerotier-one" via command line switch
* Leaking sockets on some platforms when uPnP/NAT-PMP is enabled
* Fixed two very rare multithreading issues that were only observed on certain systems
* Platform-Specific Changes
* MacOS
* Installer now loads the kernel extension right away so that High Sierra users will see the prompt to authorize it. This is done in the "Security & Privacy" preference pane and must be done directly on the console (not via remote desktop). On High Sierra and newer kexts must be authorized at the console via security settings system preferences pane.
* Windows
* The Windows installer should now install the driver without requiring a special prompt in most cases. This should make it easier for our packages to be accepted into and updated in the Chocolatey repository and should make it easier to perform remote installs across groups of machines using IT management and provisioning tools.
* The Windows official packages are now signed with an EV certificate (with hardware key).
* The Windows UI can now log into ZeroTier Central and join networks via the Central API.
* The `zerotier-idtool` command should now work on Windows without ugly hacks.
* Upgraded the installer version.
* Made a few changes to hopefully fix sporadic "will not uninstall" problems, though we cannot duplicate these issues ourselves.
* Linux
* Device names are now generated deterministically based on network IDs for all newly joined networks.
* Android
* Multicast now works on Android in most cases! Android apps can send and receive multicast and subscribe to multicast group IPs. Note that in some cases the app must bind to the specific correct interface for this to work.
* IPv6 can be disabled in UI for cases where it causes problems.
# 2017-04-20 -- Version 1.2.4
* Managed routes are now only bifurcated for the default route. This is a change in behavior, though few people will probably notice. Bifurcating all managed routes was causing more trouble than it was worth for most users.
* Up to 2X crypto speedup on x86-64 (except Windows, which will take some porting) and 32-bit ARM platforms due to integration of fast assembly language implementations of Salsa20/12 from the [supercop](http://bench.cr.yp.to/supercop.html) code base. These were written by Daniel J. Bernstein and are in the public domain. My Macbook Pro (Core i5 2.8ghz) now does almost 1.5GiB/sec Salsa20/12 per core and a Raspberry Pi got a 2X boost. 64-bit ARM support and Windows support will take some work but should not be too hard.
* Refactored code that manages credentials to greatly reduce memory use in most cases. This may also result in a small performance improvement.
* Reworked and simplified path selection and priority logic to fix path instability and dead path persistence edge cases. There have been some sporadic reports of persistent path instabilities and dead paths hanging around that take minutes to resolve. These have proven difficult to reproduce in house, but hopefully this will fix them. In any case it seems to speed up path establishment in our tests and it makes the code simpler and more readable.
* Eliminated some unused cruft from the code around path management and in the peer class.
* Fixed an issue causing build problems on some MIPS architecture systems.
* Fixed Windows forgetting routes on sleep/wake or in some other circumstances. (GitHub issue #465)
# 2017-03-17 -- Version 1.2.2
* A bug causing unreliable multicast propagation (GitHub issue #461).
* A crash in ARM binaries due to a build chain and flags problem.
* A bug in the network controller preventing members from being listed (GitHub issue #460).
# 2017-03-14 -- Version 1.2.0
Version 1.2.0 is a major milestone release representing almost nine months of work. It includes our rules engine for distributed network packet filtering and security monitoring, federated roots, and many other architectural and UI improvements and bug fixes.
@@ -13,7 +196,7 @@ The largest new feature in 1.2.0, and the product of many months of work, is our
Rules allow you to filter packets on your network and vector traffic to security observers. Security observation can be performed in-band using REDIRECT or out of band using TEE.
Tags and capabilites provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
Tags and capabilities provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
See the [rules engine announcement blog post](https://www.zerotier.com/blog/?p=927) for an in-depth discussion of theory and implementation. The [manual](https://www.zerotier.com/manual.shtml) contains detailed information on rule, tag, and capability use, and the `rule-compiler/` subfolder of the ZeroTier source tree contains a JavaScript function to compile rules in our human-readable rule definition language into rules suitable for import into a network controller. (ZeroTier Central uses this same script to compile rules on [my.zerotier.com](https://my.zerotier.com/).)
@@ -96,7 +279,7 @@ A special kind of public network called an ad-hoc network may be accessed by joi
| Start of port range (hex)
Reserved ZeroTier address prefix indicating a controller-less network
Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to desintation ports within the encoded range.
Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to destination ports within the encoded range.
For example `ff00160016000000` is an ad-hoc network allowing only SSH, while `ff0000ffff000000` is an ad-hoc network allowing any UDP or TCP port.
@@ -111,7 +294,7 @@ If you have data in an old SQLite3 controller we've included a NodeJS script in
## Major Bug Fixes in 1.2.0
* **The Windows HyperV 100% CPU bug is FINALLY DEAD**: This long-running problem turns out to have been an issue with Windows itself, but one we were triggering by placing invalid data into the Windows registry. Microsoft is aware of the issue but we've also fixed the triggering problem on our side. ZeroTier should now co-exist quite well with HyperV and should now be able to be bridged with a HyperV virtual switch.
* **Segmenation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
* **Segmentation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
* **Windows firewall blocks local JSON API**: On some Windows systems the firewall likes to block 127.0.0.1:9993 for mysterious reasons. This is now fixed in the installer via the addition of another firewall exemption rule.
* **UI crash on embedded Windows due to missing fonts**: The MSI installer now ships fonts and will install them if they are not present, so this should be fixed.
Loaded 20 of 774 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Adam Ierymenko
Joseph Henry
joseph-henry
Pablo Hinojosa
Sean OMeara
Erik Hollensbe
YI
showipintbri
Adam Ierymenko
bradleydiggs
David Nadlinger
Aaron LI
Grant Limberg
capruro
Grant Limberg
Joseph Henry
Vincent Milum Jr
Travis LaDuke
Michael Adams
Suad Halilovic
Gleb Panov
Athanasios Oikonomou
root
Dave Cottlehuber
0xflotus
wendelb
Ellen Wang
Aleksandar Topuzović
Bruno Tavares
Tommy Yang
Asbjørn Enge
Diego Schulz
Guillaume de Jabrun
Alex Forencich
Karsten Elfenbein
Jonathon Fernyhough
Elmar S. Heeb
neutronscott
Nicholas Schmidt
Dezhi “Andy” Fang
Alex
Monty A
Daniel Skowroński
