Version 1.2.2 fixes a few bugs discovered after the 1.2.0 release. These are:
* A bug causing unreliable multicast propagation (GitHub issue #461).
* A crash in ARM binaries due to a build chain and flags problem.
* A bug in the network controller preventing members from being listed (GitHub issue #460).
The largest new feature in 1.2.0, and the product of many months of work, is our advanced network rules engine. With this release we achieve traffic control, security monitoring, and micro-segmentation capability on par with many enterprise SDN solutions designed for use in advanced data centers and corporate networks.
Rules allow you to filter packets on your network and vector traffic to security observers. Security observation can be performed in-band using REDIRECT or out of band using TEE.
Tags and capabilites provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
See the [rules engine announcement blog post](https://www.zerotier.com/blog/?p=927) for an in-depth discussion of theory and implementation. The [manual](https://www.zerotier.com/manual.shtml) contains detailed information on rule, tag, and capability use, and the `rule-compiler/` subfolder of the ZeroTier source tree contains a JavaScript function to compile rules in our human-readable rule definition language into rules suitable for import into a network controller. (ZeroTier Central uses this same script to compile rules on [my.zerotier.com](https://my.zerotier.com/).)
It's now possible to create your own root servers and add them to the root server pool on your nodes. This is done by creating what's called a "moon," which is a signed enumeration of root servers and their stable points on the network. Refer to the [manual](https://www.zerotier.com/manual.shtml) for instructions.
Federated roots achieve a number of things:
* You can deploy your own infrastructure to reduce dependency on ours.
* You can deploy roots *inside your LAN* to ensure that network connectivity inside your facility still works if the Internet goes down. This is the first step toward making ZeroTier viable as an in-house SDN solution.
* Roots can be deployed inside national boundaries for countries with data residency laws or "great firewalls." (As of 1.2.0 there is still no way to force all traffic to use these roots, but that will be easy to do in a later version.)
* Last but not least this makes ZeroTier somewhat less centralized by eliminating any hard dependency on ZeroTier, Inc.'s infrastructure.
Our roots will of course remain and continue to provide zero-configuration instant-on deployment, a secure global authority for identities, and free traffic relaying for those who can't establish peer to peer connections.
An element of our design philosophy is "features are bugs." This isn't an absolute dogma but more of a guiding principle. We try as hard as we can to avoid adding features, especially "knobs" that must be tweaked by a user.
As of 1.2.0 we've decided that certain knobs are unavoidable, and so there is now a `local.conf` file that can be used to configure them. See the ZeroTier One documentation for these. They include:
* Blacklisting interfaces you want to make sure ZeroTier doesn't use for network traffic, such as VPNs, slow links, or backplanes designated for only certain kinds of traffic.
* Turning uPnP/NAT-PMP on or off.
* Configuring software updates on Windows and Mac platforms.
* Defining trusted paths (the old trusted paths file is now deprecated)
* Setting the ZeroTier main port so it doesn't have to be changed on the command line, which is very inconvenient in many cases.
A good software update system for Windows and Mac clients has been a missing feature in previous versions. It does exist but we've been shy about using it so far due to its fragility in some environments.
We've greatly improved this mechanism in 1.2.0. Not only does it now do a better job of actually invoking the update, but it also transfers updates in-band using the ZeroTier protocol. This means it can work in environments that do not allows http/https traffic or that force it through proxies. There's also now an update channel setting: `beta` or `release` (the default).
Software updates are authenticated three ways:
1. ZeroTier's own signing key is used to sign all updates and this signature is checked prior to installation. ZeroTier, Inc.'s signatures are performed on an air-gapped machine.
2. Updates for Mac and Windows are signed using Apple and Microsoft (DigiCert EV) keys and will not install unless these signatures are also valid.
3. The new in-band update mechanism also authenticates the source of the update via ZeroTier's built-in security features. This provides transport security, while 1 and 2 provide security of the update at rest.
Updates are now configurable via `local.conf`. There are three options: `disable`, `download`, and `apply`. The third (apply) is the default for official builds on Windows and Mac, making updates happen silently and automatically as they do for popular browsers like Chrome and Firefox. Updates are disabled by default on Linux and other Unix-type systems as these are typically updated through package managers.
Version 1.2.0 is now aware of the link quality of direct paths with other 1.2.0 nodes. This information isn't used yet but is visible through the JSON API. (Quality always shows as 100% with pre-1.2.0 nodes.) Quality is measured passively with no additional overhead using a counter based packet loss detection algorithm.
This information is visible from the command line via `listpeers`:
200 listpeers XXXXXXXXXX 199.XXX.XXX.XXX/9993;10574;15250;1.00 48 1.2.0 LEAF
200 listpeers XXXXXXXXXX 195.XXX.XXX.XXX/45584;467;7608;0.44 290 1.2.0 LEAF
The first peer's path is at 100% (1.00), while the second peer's path is suffering quite a bit of packet loss (0.44).
Link quality awareness is a precursor to intelligent multi-path and QoS support, which will in future versions bring us to feature parity with SD-WAN products like Cisco iWAN.
Version 1.2.0 adds anti-DOS (denial of service) rate limits and other hardening for improved resiliency against a number of denial of service attack scenarios.
It also adds a mechanism for instantaneous credential revocation. This can be used to revoke certificates of membership instantly to kick a node off a network (for private networks) and also to revoke capabilities and tags. The new controller sends revocations by default when a peer is de-authorized.
Revocations propagate using a "rumor mill" peer to peer algorithm. This means that a controller need only successfully send a revocation to at least one member of a network with connections to other active members. At this point the revocation will flood through the network peer to peer very quickly. This helps make revocations more robust in the face of poor connectivity with the controller or attempts to incapacitate the controller with denial of service attacks, as well as making revocations faster on huge networks.
The Mac has a whole new UI built natively in Objective-C. It provides a pulldown similar in appearance and operation to the Mac WiFi task bar menu.
The Windows UI has also been improved and now provides a task bar icon that can be right-clicked to manage networks. Both now expose managed route and IP permissions, allowing nodes to easily opt in to full tunnel operation if you have a router configured on your network.
A special kind of public network called an ad-hoc network may be accessed by joining a network ID with the format:
ffSSSSEEEE000000
| | | |
| | | Reserved for future use, must be 0
| | End of port range (hex)
| Start of port range (hex)
Reserved ZeroTier address prefix indicating a controller-less network
Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to desintation ports within the encoded range.
For example `ff00160016000000` is an ad-hoc network allowing only SSH, while `ff0000ffff000000` is an ad-hoc network allowing any UDP or TCP port.
Keep in mind that these networks are public and anyone in the entire world can join them. Care must be taken to avoid exposing vulnerable services or sharing unwanted files or other resources.
The network controller has been largely rewritten to use a simple in-filesystem JSON data store in place of SQLite, and it is now included by default in all Windows, Mac, Linux, and BSD builds. This means any desktop or server node running ZeroTier One can now be a controller with no recompilation needed.
If you have data in an old SQLite3 controller we've included a NodeJS script in `controller/migrate-sqlite` to migrate data to the new format. If you don't migrate, members will start getting `NOT_FOUND` when they attempt to query for updates.
* **The Windows HyperV 100% CPU bug is FINALLY DEAD**: This long-running problem turns out to have been an issue with Windows itself, but one we were triggering by placing invalid data into the Windows registry. Microsoft is aware of the issue but we've also fixed the triggering problem on our side. ZeroTier should now co-exist quite well with HyperV and should now be able to be bridged with a HyperV virtual switch.
* **Segmenation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
* **Windows firewall blocks local JSON API**: On some Windows systems the firewall likes to block 127.0.0.1:9993 for mysterious reasons. This is now fixed in the installer via the addition of another firewall exemption rule.
* **UI crash on embedded Windows due to missing fonts**: The MSI installer now ships fonts and will install them if they are not present, so this should be fixed.
* **Improved dead path detection**: ZeroTier is now more aggressive about expiring paths that do not seem to be active. If a path seems marginal it is re-confirmed before re-use.
* **Minor performance improvements**: We've reduced unnecessary memcpy's and made a few other performance improvements in the core.
* **Linux static binaries**: For our official packages (the ones in the download.zerotier.com apt and yum repositories) we now build Linux binaries with static linking. Hopefully this will stop all the bug reports relating to library inconsistencies, as well as allowing our deb packages to run on a wider variety of Debian-based distributions. (There are far too many of these to support officially!) The overhead for this is very small, especially since we built our static versions against musl-libc. Distribution maintainers are of course free to build dynamically linked versions for inclusion into distributions; this only affects our official binaries.
Merge branch 'master' of github.com:zerotier/ZeroTierOne
The largest new feature in 1.2.0, and the product of many months of work, is our advanced network rules engine. With this release we achieve traffic control, security monitoring, and micro-segmentation capability on par with many enterprise SDN solutions designed for use in advanced data centers and corporate networks.
Rules allow you to filter packets on your network and vector traffic to security observers. Security observation can be performed in-band using REDIRECT or out of band using TEE.
Tags and capabilites provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
See the [rules engine announcement blog post](https://www.zerotier.com/blog/?p=927) for an in-depth discussion of theory and implementation. The [manual](https://www.zerotier.com/manual.shtml) contains detailed information on rule, tag, and capability use, and the `rule-compiler/` subfolder of the ZeroTier source tree contains a JavaScript function to compile rules in our human-readable rule definition language into rules suitable for import into a network controller. (ZeroTier Central uses this same script to compile rules on [my.zerotier.com](https://my.zerotier.com/).)
It's now possible to create your own root servers and add them to the root server pool on your nodes. This is done by creating what's called a "moon," which is a signed enumeration of root servers and their stable points on the network. Refer to the [manual](https://www.zerotier.com/manual.shtml) for instructions.
Federated roots achieve a number of things:
* You can deploy your own infrastructure to reduce dependency on ours.
* You can deploy roots *inside your LAN* to ensure that network connectivity inside your facility still works if the Internet goes down. This is the first step toward making ZeroTier viable as an in-house SDN solution.
* Roots can be deployed inside national boundaries for countries with data residency laws or "great firewalls." (As of 1.2.0 there is still no way to force all traffic to use these roots, but that will be easy to do in a later version.)
* Last but not least this makes ZeroTier somewhat less centralized by eliminating any hard dependency on ZeroTier, Inc.'s infrastructure.
Our roots will of course remain and continue to provide zero-configuration instant-on deployment, a secure global authority for identities, and free traffic relaying for those who can't establish peer to peer connections.
An element of our design philosophy is "features are bugs." This isn't an absolute dogma but more of a guiding principle. We try as hard as we can to avoid adding features, especially "knobs" that must be tweaked by a user.
As of 1.2.0 we've decided that certain knobs are unavoidable, and so there is now a `local.conf` file that can be used to configure them. See the ZeroTier One documentation for these. They include:
* Blacklisting interfaces you want to make sure ZeroTier doesn't use for network traffic, such as VPNs, slow links, or backplanes designated for only certain kinds of traffic.
* Turning uPnP/NAT-PMP on or off.
* Configuring software updates on Windows and Mac platforms.
* Defining trusted paths (the old trusted paths file is now deprecated)
* Setting the ZeroTier main port so it doesn't have to be changed on the command line, which is very inconvenient in many cases.
A good software update system for Windows and Mac clients has been a missing feature in previous versions. It does exist but we've been shy about using it so far due to its fragility in some environments.
We've greatly improved this mechanism in 1.2.0. Not only does it now do a better job of actually invoking the update, but it also transfers updates in-band using the ZeroTier protocol. This means it can work in environments that do not allows http/https traffic or that force it through proxies. There's also now an update channel setting: `beta` or `release` (the default).
Software updates are authenticated three ways:
1. ZeroTier's own signing key is used to sign all updates and this signature is checked prior to installation. ZeroTier, Inc.'s signatures are performed on an air-gapped machine.
2. Updates for Mac and Windows are signed using Apple and Microsoft (DigiCert EV) keys and will not install unless these signatures are also valid.
3. The new in-band update mechanism also authenticates the source of the update via ZeroTier's built-in security features. This provides transport security, while 1 and 2 provide security of the update at rest.
Updates are now configurable via `local.conf`. There are three options: `disable`, `download`, and `apply`. The third (apply) is the default for official builds on Windows and Mac, making updates happen silently and automatically as they do for popular browsers like Chrome and Firefox. Updates are disabled by default on Linux and other Unix-type systems as these are typically updated through package managers.
Version 1.2.0 is now aware of the link quality of direct paths with other 1.2.0 nodes. This information isn't used yet but is visible through the JSON API. (Quality always shows as 100% with pre-1.2.0 nodes.) Quality is measured passively with no additional overhead using a counter based packet loss detection algorithm.
This information is visible from the command line via `listpeers`:
200 listpeers XXXXXXXXXX 199.XXX.XXX.XXX/9993;10574;15250;1.00 48 1.2.0 LEAF
200 listpeers XXXXXXXXXX 195.XXX.XXX.XXX/45584;467;7608;0.44 290 1.2.0 LEAF
The first peer's path is at 100% (1.00), while the second peer's path is suffering quite a bit of packet loss (0.44).
Link quality awareness is a precursor to intelligent multi-path and QoS support, which will in future versions bring us to feature parity with SD-WAN products like Cisco iWAN.
Version 1.2.0 adds anti-DOS (denial of service) rate limits and other hardening for improved resiliency against a number of denial of service attack scenarios.
It also adds a mechanism for instantaneous credential revocation. This can be used to revoke certificates of membership instantly to kick a node off a network (for private networks) and also to revoke capabilities and tags. The new controller sends revocations by default when a peer is de-authorized.
Revocations propagate using a "rumor mill" peer to peer algorithm. This means that a controller need only successfully send a revocation to at least one member of a network with connections to other active members. At this point the revocation will flood through the network peer to peer very quickly. This helps make revocations more robust in the face of poor connectivity with the controller or attempts to incapacitate the controller with denial of service attacks, as well as making revocations faster on huge networks.
The Mac has a whole new UI built natively in Objective-C. It provides a pulldown similar in appearance and operation to the Mac WiFi task bar menu.
The Windows UI has also been improved and now provides a task bar icon that can be right-clicked to manage networks. Both now expose managed route and IP permissions, allowing nodes to easily opt in to full tunnel operation if you have a router configured on your network.
A special kind of public network called an ad-hoc network may be accessed by joining a network ID with the format:
ffSSSSEEEE000000
| | | |
| | | Reserved for future use, must be 0
| | End of port range (hex)
| Start of port range (hex)
Reserved ZeroTier address prefix indicating a controller-less network
Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to desintation ports within the encoded range.
For example `ff00160016000000` is an ad-hoc network allowing only SSH, while `ff0000ffff000000` is an ad-hoc network allowing any UDP or TCP port.
Keep in mind that these networks are public and anyone in the entire world can join them. Care must be taken to avoid exposing vulnerable services or sharing unwanted files or other resources.
The network controller has been largely rewritten to use a simple in-filesystem JSON data store in place of SQLite, and it is now included by default in all Windows, Mac, Linux, and BSD builds. This means any desktop or server node running ZeroTier One can now be a controller with no recompilation needed.
If you have data in an old SQLite3 controller we've included a NodeJS script in `controller/migrate-sqlite` to migrate data to the new format. If you don't migrate, members will start getting `NOT_FOUND` when they attempt to query for updates.
* **The Windows HyperV 100% CPU bug is FINALLY DEAD**: This long-running problem turns out to have been an issue with Windows itself, but one we were triggering by placing invalid data into the Windows registry. Microsoft is aware of the issue but we've also fixed the triggering problem on our side. ZeroTier should now co-exist quite well with HyperV and should now be able to be bridged with a HyperV virtual switch.
* **Segmenation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
* **Windows firewall blocks local JSON API**: On some Windows systems the firewall likes to block 127.0.0.1:9993 for mysterious reasons. This is now fixed in the installer via the addition of another firewall exemption rule.
* **UI crash on embedded Windows due to missing fonts**: The MSI installer now ships fonts and will install them if they are not present, so this should be fixed.
* **Improved dead path detection**: ZeroTier is now more aggressive about expiring paths that do not seem to be active. If a path seems marginal it is re-confirmed before re-use.
* **Minor performance improvements**: We've reduced unnecessary memcpy's and made a few other performance improvements in the core.
* **Linux static binaries**: For our official packages (the ones in the download.zerotier.com apt and yum repositories) we now build Linux binaries with static linking. Hopefully this will stop all the bug reports relating to library inconsistencies, as well as allowing our deb packages to run on a wider variety of Debian-based distributions. (There are far too many of these to support officially!) The overhead for this is very small, especially since we built our static versions against musl-libc. Distribution maintainers are of course free to build dynamically linked versions for inclusion into distributions; this only affects our official binaries.
Move setting _initialized = true until after WindowsEthernetTap::threadMain() has actually created and brought up the adapter. Also in OneService::nodeVirtualNetworkConfigFunction(), wait up to 5 seconds for WindowsEthernatTap::isInitialized() to return true before attempting to configure the interface and managed routes. Without this, the adapter doesnt actually exist yet when trying to add routes
Also, dont add the prefix length to the IP addresses in there.
Appears to fix Github Issue #357. Still waiting on some info from Microsoft for confirmation of a few things.
WindowsEthernetTap was storing the MTU for the ZeroTier Adapter as a
DWORD. When changing the value from the network adapter settings UI, it
was stored as a string. Updated the call when creating the adapter to
store it as a string as windows expects.
- ZT will only drop root privileges if zerotier-one user exists. It is created by
Debian postinst script - in other cases the user has to be created by administrator.
- Linux >=4.3 with ambient capabilities is required, otherwise ZT will silently
- "-U" option now also disables privileges dropping
Version 1.1.10 is a minor bug fix release. It fixes the following two issues:
* Routes are no longer added 'via' IP addresses of the current node's ZT interface.
* IPv6 default route is now fixed on Mac.
* IP addresses are now deleted before new ones are added to allow changes to netmask
on Mac and some other platforms.
This version also quietly introduces a feature called trusted paths. We'll be
documenting this in our community section soon. This is an experimental feature
for in-LAN SDN use.
1.1.8 is a very minor update. It contains a small fix to IP address management
on Linux but other than that there are no changes to the regular ZeroTier One
code.
The 1.1.8 controller contains several fixes and improvements:
* Network authorizedMemberCount works again
* Managed IP addresses are pushed as long as they fall within a LAN-local managed
route. The v4AssignMode and v6AssignMode settings only control automatic IP
assignment from pools. In previous versions v4AssignMode had to be on for IPs
to be pushed at all, making it impossible to just manually assign IPs.
* IPv6 now has an auto-assign mode ('zt' to match V4) that can work alongside the
special RFC4193 and 6PLANE modes (though it is not NDP emulated). If this is
enabled IPv6 addresses will be auto-assigned from IPv6 ranges in the assignment
pool list.
* Controller documentation is MUCH better! See controller/README.md for actual
useful information!
Finally 1.1.8 has several improvements to linux-build-farm including Debian wheezy
support.
(Retagged due to silly one-line bug and left over debug line... doh!)
Version 1.1.6 contains several significant improvements for use in complex network
environments along with some minor bug fixes and improvements to path stability and
dead path detection.
ROUTE MANAGEMENT AND FULL TUNNEL SUPPORT
1.1.6 is the first version of ZeroTier One to permit "full tunnel" (default route
override) operation on Linux, Mac, and Windows. This allows all Internet traffic
to be tunneled through ZeroTier while allowing ZeroTier peer-to-peer traffic to
continue to use the physical interface. 1.1.6 also brings route management support
and permissions settings for local networks to control whether networks are allowed
to modify the routing table or override default routing.
This is currently considered a beta/experimental feature and must be enabled via
the command line interface.
Route management and default route override requires support at the network controller.
When my.zerotier.com is updated and ready, we will post more information and testing
instructions at: https://www.zerotier.com/community
HIGHLY SCALABLE CONTAINER NETWORKING
1.1.6 also brings a new multicast-free (NDP emulated) IPv6 private addressing scheme
called "6plane." 6plane provides each host with a private IPv6 /80 and routes *all*
IPv6 traffic for this subnet to the host via transparent NDP emulation. This /80 can
then be assigned to Docker or other container/VM managers to assign a network-wide
IPv6 /128 to every container. Since NDP is emulated and multicast isn't needed, this
system can scale to millions of containers or more on a single backplane network with
a high degree of efficiency and reliability.
6plane also requires controller support. Look for it at my.zerotier.com once we have
upgraded our core infrastructure and web UIs.
(All hosts must be running 1.1.6 for 6plane to work properly. Other IPv6 addresses
or addressing modes are not affected and normal IPv6 NDP will continue to work
alongside 6plane in the same network.)
OTHER CHANGES
* Upgraded bundled miniupnpc, libnatpmp, and http-parser.
* New Debian and RPM packaging that is closer to compliance with distribution
guidelines, and a new Dockerized Linux package build system in linux-build-farm/
that can build every package on actual images of the correct distribution.
* Improvements to dead path detection.
* IPv6 now uses keepalive because a significant number of stateful IPv6 edge
routers have very short timeouts (30 seconds or less!).
* Significant performance improvements to network controllers under high load.
* Enable -fstack-protector-strong for better stack canary (security) support
in binaries. Note that this may require newer gcc/g++ or clang.
COMING SOON
The next version of ZeroTier One should have a new Mac UI. It's a system tray app
that looks and behaves a lot like the Mac WiFi pulldown menu. We'll also be adding
GUI support for default route and route management options and other new features.
Shortly after that we plan on adding full OpenFlow-like SDN rules engine support
to the ZeroTier core, making our planetary Ethernet switch a fully manageable smart
switch and enabling sophisticated security and flow rule management.
Version 1.1.6 contains several significant improvements for use in complex network
environments along with some minor bug fixes and improvements to path stability and
dead path detection.
ROUTE MANAGEMENT AND FULL TUNNEL SUPPORT
1.1.6 is the first version of ZeroTier One to permit "full tunnel" (default route
override) operation on Linux, Mac, and Windows. This allows all Internet traffic
to be tunneled through ZeroTier while allowing ZeroTier peer-to-peer traffic to
continue to use the physical interface. 1.1.6 also brings route management support
and permissions settings for local networks to control whether networks are allowed
to modify the routing table or override default routing.
This is currently considered a beta/experimental feature and must be enabled via
the command line interface.
Route management and default route override requires support at the network controller.
When my.zerotier.com is updated and ready, we will post more information and testing
instructions at: https://www.zerotier.com/community
HIGHLY SCALABLE CONTAINER NETWORKING
1.1.6 also brings a new multicast-free (NDP emulated) IPv6 private addressing scheme
called "6plane." 6plane provides each host with a private IPv6 /80 and routes *all*
IPv6 traffic for this subnet to the host via transparent NDP emulation. This /80 can
then be assigned to Docker or other container/VM managers to assign a network-wide
IPv6 /128 to every container. Since NDP is emulated and multicast isn't needed, this
system can scale to millions of containers or more on a single backplane network with
a high degree of efficiency and reliability.
6plane also requires controller support. Look for it at my.zerotier.com once we have
upgraded our core infrastructure and web UIs.
(All hosts must be running 1.1.6 for 6plane to work properly. Other IPv6 addresses
or addressing modes are not affected and normal IPv6 NDP will continue to work
alongside 6plane in the same network.)
OTHER CHANGES
* Upgraded bundled miniupnpc, libnatpmp, and http-parser.
* New Debian and RPM packaging that is closer to compliance with distribution
guidelines, and a new Dockerized Linux package build system in linux-build-farm/
that can build every package on actual images of the correct distribution.
* Improvements to dead path detection.
* IPv6 now uses keepalive because a significant number of stateful IPv6 edge
routers have very short timeouts (30 seconds or less!).
* Significant performance improvements to network controllers under high load.
* Enable -fstack-protector-strong for better stack canary (security) support
in binaries. Note that this may require newer gcc/g++ or clang.
COMING SOON
The next version of ZeroTier One should have a new Mac UI. It's a system tray app
that looks and behaves a lot like the Mac WiFi pulldown menu. We'll also be adding
GUI support for default route and route management options and other new features.
Shortly after that we plan on adding full OpenFlow-like SDN rules engine support
to the ZeroTier core, making our planetary Ethernet switch a fully manageable smart
switch and enabling sophisticated security and flow rule management.
(and other host friendly) IPv6 /80 magic subnetting to allow massive
multicast-free NDP emulated IPv6 networks where each host can have a
/48 worth of IPv6 IPs for internal containers, VMs, etc.
Alan Kay, thou art avenged.
https://ivanovivan.wordpress.com/2010/09/13/alan-kay-quotes/
implementing NAT, do implement stateful firewalling with absurdly short
timeouts (<60s). Keepalives are still required in IPv6.
Network engineers continue to mindlessly carry forward cruft and baggage
from IPv4 to IPv6.
Many NATs revert to symmetric behavior from friendlier modes if they cannot
preserve ports. This can occur if there is, for example, more than one ZT
device behind the NAT using port 9993.
Others (Airport Extreme?) seem to have bugs in which they completely freak
out if more than one device tries to do a lot of mappings using the same
internal local port.
Mostly to fix the latter case and somewhat to fix the former, we introduce
a secondary port. ZeroTier now binds 9993 (or whatever port you specify)
plus another port computed deterministically from your ZeroTier address.
This port is used for new links 1/4 of the time.
This mostly addresses the second problem above and partly helps to address
the first.
If uPnP/NAT-PMP is enabled we also still open a tertiary port because some
routers freak out if NAT-t is attempted using the same port as uPnP.
All of this is IPv4 only of course. IPv6 is sane.
ZeroTier One version 1.1.4 has several improvements to path stability and
path learning logic. It also contains the BETA version of Network Containers!
See netcon/README.md and https://www.zerotier.com/product-netcon.shtml for more
information.
Improvements in 1.1.4:
- Paths are now confirmed more aggressively on inactivity. This version can
usually detect a dead direct path in under 10 seconds vs 1-2 minutes in
previous versions.
- Logic to prevent recursion has been overhauled. The old versions had code
that explicitly checked packet contents (AntiRecursion.hpp). This is now
gone in favor of a function that checks IP addresses of ZeroTier devices
when paths are learned or probed. This is both more efficient and less
prone to edge case failures. Testing in several complex environments has
shown that this eliminates a few edge cases where inappropriate paths would
be used.
* ZeroTier protocol design and core network virtualization engine, ZeroTier One service, React web UI, packaging for most platforms, kitchen sink...<br>
* ZeroTier Core and ZeroTier One virtual networking service<br>
Adam Ierymenko / adam.ierymenko@zerotier.com
* Java JNI Interface to enable Android application development, and Android app itself (code for that is elsewhere)<br>
Grant Limberg / glimberg@gmail.com
## Contributors
* ZeroTier SDK (formerly known as Network Containers)<br>
Joseph Henry / joseph.henry@zerotier.com
## Third Party Contributors
* A number of fixes and improvements to the new controller, other stuff.<br>
Kees Bos / https://github.com/keesbos
Kees Bos / https://github.com/keesbos/
* Debugging and testing, OpenWRT support fixes.<br>
Moritz Warning / moritzwarning@web.de
* Debian GNU/Linux packaging, manual pages, and license compliance edits.<br>
Ben Finney <ben+zerotier@benfinney.id.au>
* Several others made smaller contributions, which GitHub tracks here:<br>
digital signature algorithm, and Poly1305 MAC algorithm, all by
Daniel J. Bernstein (public domain)<br>
http://cr.yp.to/
*C++11 json (nlohmann/json) by Niels Lohmann
*MiniUPNPC by Thomas Bernard [BSD]
http://miniupnp.free.fr
*Files: ext/json/*
* Home page: https://github.com/nlohmann/json
* License grant: MIT
* TunTapOSX by Mattias Nissler
* Files: ext/tap-mac/tuntap/*
* Home page: http://tuntaposx.sourceforge.net/
* License grant: BSD attribution no-endorsement
* ZeroTier Modifications: change interface name to zt#, increase max MTU, increase max devices
* tap-windows6 by the OpenVPN project
* Files: windows/TapDriver6/*
* Home page: https://github.com/OpenVPN/tap-windows6/
* License grant: GNU GPL v2
* ZeroTier Modifications: change name of driver to ZeroTier, add ioctl() to get L2 multicast memberships (source is in ext/ and modifications inherit GPL)
* Salsa20 stream cipher, Curve25519 elliptic curve cipher, Ed25519 digital signature algorithm, and Poly1305 MAC algorithm, all by Daniel J. Bernstein
The final .AIP file can only be edited on Windows with [Advanced Installer Enterprise](http://www.advancedinstaller.com/). In addition to incrementing the version be sure that a new product code is generated. (The "upgrade code" GUID on the other hand must never change.)
# Building for Supported Platforms
## Macintosh
Mac's easy. Just type:
make official
You will need [Packages](http://s.sudre.free.fr/Software/Packages/about.html) and our release signing key in the keychain.
## Linux
Mount the GPG key for *contact@zerotier.com* and then on an x86_64 box with a recent version of Docker and an Internet connection run:
make distclean
cd linux-build-farm
./build.sh
This will build i386 and x86_64 packages. Now ssh into our build Raspberry Pi and type `make debian` there to build the Raspbian armhf package. Copy it to `debian-jessie/` inside `linux-build-farm` so that it will be included in the repositories we generate. Now generate the YUM and APT repos:
rm -rf ~/.aptly*
rm -rf /tmp/zt-rpm-repo
./make-apt-repos.sh
./make-rpm-repos.sh
This will require the passphrase for *contact@zerotier.com*.
The contents of ~/.aptly/public must be published as `debian/` on `download.zerotier.com`. The contents of /tmp/zt-rpm-repo are published as `redhat/` on same.
## Windows
First load the Visual Studio solution and rebuild the UI and ZeroTier One in both x64 and i386 `Release` mode. Then load [Advanced Installer Enterprise](http://www.advancedinstaller.com/), check that the version is correct, and build. The build will fail if any build artifacts are missing, and Windows must have our product singing key (from DigiCert) available to sign the resulting MSI file. The MSI must then be tested on at least a few different CLEAN Windows VMs to ensure that the installer is valid and properly signed.
*After the MSI is published to download.zerotier.com in the proper RELEASE/#.#.#/dist subfolder for its version* the Chocolatey package must be rebuilt and published. Open a command prompt, change to `ext/installfiles/windows/chocolatey`, and type `choco pack`. Then use `choco push` to push it to Chocolatey (API key required).
Note that this does not cover rebuilding the drivers or their containing MSI projects, as this is typically not necessary and they are shipped in binary form in the repository for convenience.
## iOS, Android
... no docs here yet since this is done entirely out of band with regular installs.
ZeroTier is a software defined networking layer for Earth.
ZeroTier is an enterprise Ethernet switch for planet Earth.
It can be used for on-premise network virtualization, as a peer to peer VPN for mobile teams, for hybrid or multi-data-center cloud deployments, or just about anywhere else secure software defined virtual networking is useful.
It erases the LAN/WAN distinction and makes VPNs, tunnels, proxies, and other kludges arising from the inflexible nature of physical networks obsolete. Everything is encrypted end-to-end and traffic takes the most direct (peer to peer) path available.
ZeroTier One is our OS-level client service. It allows Mac, Linux, Windows, FreeBSD, and soon other types of clients to join ZeroTier virtual networks like conventional VPNs or VLANs. It can run on native systems, VMs, or containers (Docker, OpenVZ, etc.).
Visit [ZeroTier's site](https://www.zerotier.com/) for more information and [pre-built binary packages](https://www.zerotier.com/download.shtml). Apps for Android and iOS are available for free in the Google Play and Apple app stores.
Visit [ZeroTier's site](https://www.zerotier.com/) for more information. You can also download professionally packaged binary installers/packages for a variety of supported OSes there if you don't want to build ZeroTier One from source.
### Getting Started
### Building from Source
ZeroTier's basic operation is easy to understand. Devices have 10-digit *ZeroTier addresses* like `89e92ceee5` and networks have 16-digit network IDs like `8056c2e21c000001`. All it takes for a device to join a network is its 16-digit ID, and all it takes for a network to authorize a device is its 10-digit address. Everything else is automatic.
For Mac, Linux, and BSD, just type "make" (or "gmake" on BSD). You won't need much installed; here are the requirements for various platforms:
A "device" in our terminology is any "unit of compute" capable of talking to a network: desktops, laptops, phones, servers, VMs/VPSes, containers, and even user-space applications via our [SDK](https://github.com/zerotier/ZeroTierSDK).
* Mac: Xcode command line tools, and [Packages](http://s.sudre.free.fr/Software/Packages/about.html) if you want to build an OSX .pkg installer ("make mac-dist-pkg"). It should build on OSX 10.7 or newer.
* Linux: gcc/g++ or clang/clang++ (Makefile will use clang by default if available.)
* FreeBSD (and other BSD): C++ compiler (G++ usually) and GNU make (gmake).
For testing purposes we provide a public virtual network called *Earth* with network ID `8056c2e21c000001`. You can join it with:
Each supported platform has its own *make-XXX.mk* file that contains the actual make rules for the platform. The right .mk file is included by the main Makefile based on the GNU make *OSTYPE* variable. Take a look at the .mk file for your platform for other targets, debug build rules, etc.
sudo zerotier-cli join 8056c2e21c000001
Windows, of course, is special. We build for Windows with Microsoft Visual Studio 2012 on Windows 7. A solution file is located in the*windows*subfolder. Newer versions of Visual Studio (and Windows) may work but haven't been tested. Older versions almost certainly will not, since they lack things like *stdint.h* and certain STL features. MinGW or other ports of gcc/clang to Windows should also work but haven't been tested. Build steps for Windows are a bit more complicated. For the moment you are on your own there.
Now wait about 30 seconds and check your system with `ip addr list` or `ifconfig`. You'll see a new interface whose name starts with *zt*and it should quickly get an IPv4 and an IPv6 address. Once you see it get an IP, try pinging `earth.zerotier.net` at `29.209.112.93`. If you've joined Earth from more than one system, try pinging your other machine. If you don't want to belong to a giant Ethernet party line anymore, just type:
Mobile versions are in progress. They don't work yet, and in any case only the glue code will be included in this repository. The full mobile apps are in private repositories on our own git server.
sudo zerotier-cli leave 8056c2e21c000001
### Supported Platforms
The *zt* interface will disappear. You're no longer on the network.
CPU architecture shouldn't matter unless it's smaller than 32-bit or something really bizarre like a "middle-endian" processor. We have reports of ZeroTier One running on arm32, arm64, and MIPS. It builds and runs out of the box on Raspberry Pi, BeagleBone, BananaPi, and other ARM-based developer/hobbyist boards.
To create networks of your own, you'll need a network controller. ZeroTier One (for desktops and servers) includes controller functionality in its default build that can be configured via its JSON API (see [README.md in controller/](controller/)). ZeroTier provides a hosted solution with a nice web UI and SaaS add-ons at [my.zerotier.com](https://my.zerotier.com/). Basic controller functionality is free for up to 100 devices.
ZeroTier is written in C and C++ (C++03 / ISO/IEC 14882:2003) and uses data structures and algorithms from the C++03 STL. We do not use any C++11 features (yet), since we want to support a few old and embedded platforms that don't have C++11 compilers. You *will* require a compiler and headers new enough to support 64-bit integers (long long) and the *stdint.h* header. The latter could also be faked by adding defines for things like *uint32\_t*, *int64\_t*, etc.
### Project Layout
Typing "make selftest" will build a *zerotier-selftest* binary which unit tests various internals and reports on a few aspects of the build environment. It's a good idea to try this on novel platforms or architectures.
-`artwork/`: icons, logos, etc.
-`attic/`: old stuff and experimental code that we want to keep around for reference.
-`controller/`: the reference network controller implementation, which is built and included by default on desktop and server build targets.
-`debian/`: files for building Debian packages on Linux.
-`doc/`: manual pages and other documentation.
-`ext/`: third party libraries, binaries that we ship for convenience on some platforms (Mac and Windows), and installation support files.
-`include/`: include files for the ZeroTier core.
-`java/`: a JNI wrapper used with our Android mobile app. (The whole Android app is not open source but may be made so in the future.)
-`macui/`: a Macintosh menu-bar app for controlling ZeroTier One, written in Objective C.
-`node/`: the ZeroTier virtual Ethernet switch core, which is designed to be entirely separate from the rest of the code and able to be built as a stand-alone OS-independent library. Note to developers: do not use C++11 features in here, since we want this to build on old embedded platforms that lack C++11 support. C++11 can be used elsewhere.
-`osdep/`: code to support and integrate with OSes, including platform-specific stuff only built for certain targets.
-`service/`: the ZeroTier One service, which wraps the ZeroTier core and provides VPN-like connectivity to virtual networks for desktops, laptops, servers, VMs, and containers.
-`tcp-proxy/`: TCP proxy code run by ZeroTier, Inc. to provide TCP fallback (this will die soon!).
-`windows/`: Visual Studio solution files, Windows service code for ZeroTier One, and the Windows task bar app UI.
The base path contains the ZeroTier One service main entry point (`one.cpp`), self test code, makefiles, etc.
### Build and Platform Notes
To build on Mac and Linux just type `make`. On FreeBSD and OpenBSD `gmake` (GNU make) is required and can be installed from packages or ports. For Windows there is a Visual Studio solution in `windows/'.
- **Mac**
- Xcode command line tools for OSX 10.7 or newer are required.
- Tap device driver kext source is in `ext/tap-mac` and a signed pre-built binary can be found in `ext/bin/tap-mac`. You should not need to build it yourself. It's a fork of [tuntaposx](http://tuntaposx.sourceforge.net) with device names changed to `zt#`, support for a larger MTU, and tun functionality removed.
- **Linux**
- The minimum compiler versions required are GCC/G++ 4.9.3 or CLANG/CLANG++ 3.4.2.
- Linux makefiles automatically detect and prefer clang/clang++ if present as it produces smaller and slightly faster binaries in most cases. You can override by supplying CC and CXX variables on the make command line.
- CentOS 7 ships with a version of GCC/G++ that is too old, but a new enough version of CLANG can be found in the *epel* repositories. Type `yum install epel-release` and then `yum install clang` to build there.
- **Windows**
- Windows 7 or newer (and equivalent server versions) are supported. This *may* work on Vista but you're on your own there. Windows XP is not supported since it lacks many important network API functions.
- We build with Visual Studio 2015. Older versions may not work with the solution file and project files we ship and may not have new enough C++11 support.
- Pre-built signed Windows drivers are included in `ext/bin/tap-windows-ndis6`. The MSI files found there will install them on 32-bit and 64-bit systems. (These are included in our multi-architecture installer as chained MSIs.)
- Windows builds are more painful in general than other platforms and are for the adventurous.
- **FreeBSD**
- Tested most recently on FreeBSD-11. Older versions may work but we're not sure.
- GCC/G++ 4.9 and gmake are required. These can be installed from packages or ports. Type `gmake` to build.
- **OpenBSD**
- There is a limit of four network memberships on OpenBSD as there are only four tap devices (`/dev/tap0` through `/dev/tap3`). We're not sure if this can be increased.
- OpenBSD lacks `getifmaddrs` (or any equivalent method) to get interface multicast memberships. As a result multicast will only work on OpenBSD for ARP and NDP (IP/MAC lookup) and not for other purposes.
- Only tested on OpenBSD 6.0. Older versions may not work.
- GCC/G++ 4.9 and gmake are required and can be installed using `pkg_add` or from ports. They get installed in `/usr/local/bin` as `egcc` and `eg++` and our makefile is pre-configured to use them on OpenBSD.
Typing `make selftest` will build a *zerotier-selftest* binary which unit tests various internals and reports on a few aspects of the build environment. It's a good idea to try this on novel platforms or architectures.
### Running
@@ -45,10 +85,10 @@ The service is controlled via the JSON API, which by default is available at 127
Here's where home folders live (by default) on each OS:
* Linux: /var/lib/zerotier-one
* BSD: /var/db/zerotier-one
* Mac: /Library/Application Support/ZeroTier/One
*Windows: \\ProgramData\\ZeroTier\\One (That's for Windows 7. The base 'shared app data' folder might be different on different Windows versions.)
* **Windows**: `\ProgramData\ZeroTier\One` (That's for Windows 7. The base 'shared app data' folder might be different on different Windows versions.)
Running ZeroTier One on a Mac is the same, but OSX requires a kernel extension. We ship a signed binary build of the ZeroTier tap device driver, which can be installed on Mac with:
@@ -56,107 +96,17 @@ Running ZeroTier One on a Mac is the same, but OSX requires a kernel extension.
This will create the home folder for Mac, place *tap.kext* there, and set its modes correctly to enable ZeroTier One to manage it with *kextload* and *kextunload*.
We recommend using our binary packages on Windows, since there are several prerequisites such as a tap driver that must be installed on the system *and* in the home folder.
### Joining A Network
ZeroTier virtual networks are identified by 16-digit hexadecimal network IDs, while devices are identified by 10-digit addresses. To get your address run:
sudo zerotier-cli status
(Use ./zerotier-cli if you're running it right from your build folder.)
You should see something like:
200 info ########## ONLINE #.#.#
That 10-digit hex code is you. It's derived via a one-way proof of work function from your cryptographic public key. Your public key can be found in *identity.public* in ZeroTier's home folder, while *identity.secret* contains your full identity including the secret portion of the key pair.
(The identity files define your device's *identity*. Moving them to another system will move that identity. Be careful when cloning virtual machines that have identities stored on them. If two devices have the same identity, they'll "fight" over it and you won't know which device will receive network packets.)
If you want to do a quick test, you can join [Earth](https://www.zerotier.com/earth.html). It's a global public network that anyone can join. Type:
sudo zerotier-cli join 8056c2e21c000001
Then:
sudo zerotier-cli listnetworks
At first it'll be in *REQUESTING\_CONFIGURATION* state. In a few seconds to a minute you should see something like:
200 listnetworks 8056c2e21c000001 earth.zerotier.net ##:##:##:##:##:## OK PUBLIC zt0 ##.##.##.##/##
Earth will assign you an IP address in the "unofficially available" globally unrouted 28.0.0.0/7 IP block so as to avoid conflicts with local networks. (Your networks can use any IP scheme, or can even leave IP addresses unmanaged.) Once you get an IP, you should be able to ping something:
ping earth.zerotier.net
Go to [http://earth.zerotier.net/](http://earth.zerotier.net/) to see a short little welcome page that will tell you your IP and Ethernet MAC address.
Earth is a public place. If you don't want to stick around run:
sudo zerotier-cli leave 8056c2e21c000001
The network (and associated interface) should be gone.
Networks are created and administrated by network controllers. Most users will want to use our hosted controllers. Visit [our web site](https://www.zerotier.com/) for more information. Later in this README there are brief instructions about building ZeroTier One with network controller support for those who want to try running their own.
Macintosh and Windows installers also install a GUI application.
### Installing
We don't have a "make install" rule quite yet. On Linux you can type:
make installer
This will build a binary that, when run, will install ZeroTier One on most current Linux distributions. We also have RPM and DEB build files in *ext/installfiles/linux* that wrap this installer in packages for RedHat/CentOS and Debian/Ubuntu derived distributions. If *rpmbuild* is present on the system, the RPM will be built. If *dpkg-deb* is present, the DEB package will be built.
On Mac the best way is to install [Packages](http://s.sudre.free.fr/Software/Packages/about.html) and use:
make mac-dist-pkg
This builds a .pkg file that can be installed.
In FreeBSD there is now an official .pkg in the FreeBSD repository. Type "pkg install zerotier". It can also be built and installed from source.
Linux/BSD and Mac installations have an *uninstall.sh* file in their ZeroTier home folder that cleanly removes ZeroTier One from the system. Run this with:
sudo /path/to/ZeroTier/home/folder/uninstall.sh
Windows installers are insane. We build our .MSI installers with [Advanced Installer Enterprise](http://www.advancedinstaller.com). The Advanced Installer project file is in *ext/installfiles/windows*. To avoid lasting psychological trauma we recommend leaving Windows installers alone and using the pre-built Windows binaries on our web site.
### Using ZeroTier One in Docker Containers
To run the ZeroTier One service in a Docker container, run it with "--device=/dev/net/tun --cap-add=NET_ADMIN". This will allow ZeroTier One to open a "tap" virtual network port inside the container.
Alternately, you can use Ethernet bridging to bridge the *docker0* device on your system to a ZeroTier virtual network. This allows you to run ZeroTier One on the host and bridge the entire Docker network backplane to a virtual network or other hosts.
We're working on better "official" Docker support. In the meantime there is a [user-contributed project here](https://github.com/davide/docker-zerotier).
### Building with Network Controller Support
**Warning: as of beta version 1.0.3 the new network controller is not heavily tested. We recommend waiting for 1.0.4 to deploy this in production.**
Network controllers are nodes responsible for issuing configurations and certificates to members of ZeroTier virtual networks. Most users won't need to run their own, so this code is by default not included in the ZeroTier One binary.
You can build a network controller on Linux or Mac with:
make ZT_ENABLE_NETWORK_CONTROLLER=1
This will build a version that contains the Sqlite-backed network controller and associated extensions to the JSON local service control API. You will need the development headers for sqlite3 installed. On Mac these ship as part of Xcode, while on Linux they'll be found in packages for the various distributions.
See the JSON API documentation in [service/](service/) for more information about how to control controllers.
### Troubleshooting
For most users, it just works.
If you are running a local system firewall, we recommend adding a rule permitting UDP port 9993 inbound and outbound. If you installed binaries for Windows this should be done automatically. Other platforms might require manual editing of local firewall rules depending on your configuration.
The Mac firewall can be founder under "Security" in System Preferences. Linux has a variety of firewall configuration systems and tools. If you're using Ubuntu's *ufw*, you can do this:
The Mac firewall can be found under "Security" in System Preferences. Linux has a variety of firewall configuration systems and tools. If you're using Ubuntu's *ufw*, you can do this:
sudo ufw allow 9993/udp
On CentOS check */etc/sysconfig/iptables* for IPTables rules. For other distributions consult your distribution's documentation. You'll also have to check the UIs or documentation for commercial third party firewall applications like Little Snitch (Mac), McAfee Firewall Enterprise (Windows), etc. if you are running any of those. Some corporate environments might have centrally managed firewall software, so you might also have to contact IT.
On CentOS check `/etc/sysconfig/iptables` for IPTables rules. For other distributions consult your distribution's documentation. You'll also have to check the UIs or documentation for commercial third party firewall applications like Little Snitch (Mac), McAfee Firewall Enterprise (Windows), etc. if you are running any of those. Some corporate environments might have centrally managed firewall software, so you might also have to contact IT.
ZeroTier One peers will automatically locate each other and communicate directly over a local wired LAN *if UDP port 9993 inbound is open*. If that port is filtered, they won't be able to see each others' LAN announcement packets. If you're experiencing poor performance between devices on the same physical network, check their firewall settings. Without LAN auto-location peers must attempt "loopback" NAT traversal, which sometimes fails and in any case requires that every packet traverse your external router twice.
@@ -168,7 +118,7 @@ If a firewall between you and the Internet blocks ZeroTier's UDP traffic, you wi
### Contributing
There are three main branches: **edge**, **test**, and **master**. Other branches may be for specific features, tests, or use cases. In general **edge** is "bleeding" and may or may not work, while **test** should be relatively stable and **master** is the latest tagged release. Pull requests should generally be done against **test** or **edge**, since pull requests against **master** may be working against a branch that is somewhat out of date.
Please make pull requests against the `dev` branch. The `master` branch is release, and `edge` is for unstable and work in progress changes and is not likely to work.
Version 1.2.2 fixes a few bugs discovered after the 1.2.0 release. These are:
* A bug causing unreliable multicast propagation (GitHub issue #461).
* A crash in ARM binaries due to a build chain and flags problem.
* A bug in the network controller preventing members from being listed (GitHub issue #460).
------
# 2017-03-14 -- Version 1.2.0
Version 1.2.0 is a major milestone release representing almost nine months of work. It includes our rules engine for distributed network packet filtering and security monitoring, federated roots, and many other architectural and UI improvements and bug fixes.
## New Features in 1.2.0
### The ZeroTier Rules Engine
The largest new feature in 1.2.0, and the product of many months of work, is our advanced network rules engine. With this release we achieve traffic control, security monitoring, and micro-segmentation capability on par with many enterprise SDN solutions designed for use in advanced data centers and corporate networks.
Rules allow you to filter packets on your network and vector traffic to security observers. Security observation can be performed in-band using REDIRECT or out of band using TEE.
Tags and capabilites provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
See the [rules engine announcement blog post](https://www.zerotier.com/blog/?p=927) for an in-depth discussion of theory and implementation. The [manual](https://www.zerotier.com/manual.shtml) contains detailed information on rule, tag, and capability use, and the `rule-compiler/` subfolder of the ZeroTier source tree contains a JavaScript function to compile rules in our human-readable rule definition language into rules suitable for import into a network controller. (ZeroTier Central uses this same script to compile rules on [my.zerotier.com](https://my.zerotier.com/).)
### Root Server Federation
It's now possible to create your own root servers and add them to the root server pool on your nodes. This is done by creating what's called a "moon," which is a signed enumeration of root servers and their stable points on the network. Refer to the [manual](https://www.zerotier.com/manual.shtml) for instructions.
Federated roots achieve a number of things:
* You can deploy your own infrastructure to reduce dependency on ours.
* You can deploy roots *inside your LAN* to ensure that network connectivity inside your facility still works if the Internet goes down. This is the first step toward making ZeroTier viable as an in-house SDN solution.
* Roots can be deployed inside national boundaries for countries with data residency laws or "great firewalls." (As of 1.2.0 there is still no way to force all traffic to use these roots, but that will be easy to do in a later version.)
* Last but not least this makes ZeroTier somewhat less centralized by eliminating any hard dependency on ZeroTier, Inc.'s infrastructure.
Our roots will of course remain and continue to provide zero-configuration instant-on deployment, a secure global authority for identities, and free traffic relaying for those who can't establish peer to peer connections.
### Local Configuration
An element of our design philosophy is "features are bugs." This isn't an absolute dogma but more of a guiding principle. We try as hard as we can to avoid adding features, especially "knobs" that must be tweaked by a user.
As of 1.2.0 we've decided that certain knobs are unavoidable, and so there is now a `local.conf` file that can be used to configure them. See the ZeroTier One documentation for these. They include:
* Blacklisting interfaces you want to make sure ZeroTier doesn't use for network traffic, such as VPNs, slow links, or backplanes designated for only certain kinds of traffic.
* Turning uPnP/NAT-PMP on or off.
* Configuring software updates on Windows and Mac platforms.
* Defining trusted paths (the old trusted paths file is now deprecated)
* Setting the ZeroTier main port so it doesn't have to be changed on the command line, which is very inconvenient in many cases.
### Improved In-Band Software Updates
A good software update system for Windows and Mac clients has been a missing feature in previous versions. It does exist but we've been shy about using it so far due to its fragility in some environments.
We've greatly improved this mechanism in 1.2.0. Not only does it now do a better job of actually invoking the update, but it also transfers updates in-band using the ZeroTier protocol. This means it can work in environments that do not allows http/https traffic or that force it through proxies. There's also now an update channel setting: `beta` or `release` (the default).
Software updates are authenticated three ways:
1. ZeroTier's own signing key is used to sign all updates and this signature is checked prior to installation. ZeroTier, Inc.'s signatures are performed on an air-gapped machine.
2. Updates for Mac and Windows are signed using Apple and Microsoft (DigiCert EV) keys and will not install unless these signatures are also valid.
3. The new in-band update mechanism also authenticates the source of the update via ZeroTier's built-in security features. This provides transport security, while 1 and 2 provide security of the update at rest.
Updates are now configurable via `local.conf`. There are three options: `disable`, `download`, and `apply`. The third (apply) is the default for official builds on Windows and Mac, making updates happen silently and automatically as they do for popular browsers like Chrome and Firefox. Updates are disabled by default on Linux and other Unix-type systems as these are typically updated through package managers.
### Path Link Quality Awareness
Version 1.2.0 is now aware of the link quality of direct paths with other 1.2.0 nodes. This information isn't used yet but is visible through the JSON API. (Quality always shows as 100% with pre-1.2.0 nodes.) Quality is measured passively with no additional overhead using a counter based packet loss detection algorithm.
This information is visible from the command line via `listpeers`:
The first peer's path is at 100% (1.00), while the second peer's path is suffering quite a bit of packet loss (0.44).
Link quality awareness is a precursor to intelligent multi-path and QoS support, which will in future versions bring us to feature parity with SD-WAN products like Cisco iWAN.
### Security Improvements
Version 1.2.0 adds anti-DOS (denial of service) rate limits and other hardening for improved resiliency against a number of denial of service attack scenarios.
It also adds a mechanism for instantaneous credential revocation. This can be used to revoke certificates of membership instantly to kick a node off a network (for private networks) and also to revoke capabilities and tags. The new controller sends revocations by default when a peer is de-authorized.
Revocations propagate using a "rumor mill" peer to peer algorithm. This means that a controller need only successfully send a revocation to at least one member of a network with connections to other active members. At this point the revocation will flood through the network peer to peer very quickly. This helps make revocations more robust in the face of poor connectivity with the controller or attempts to incapacitate the controller with denial of service attacks, as well as making revocations faster on huge networks.
### Windows and Macintosh UI Improvements (ZeroTier One)
The Mac has a whole new UI built natively in Objective-C. It provides a pulldown similar in appearance and operation to the Mac WiFi task bar menu.
The Windows UI has also been improved and now provides a task bar icon that can be right-clicked to manage networks. Both now expose managed route and IP permissions, allowing nodes to easily opt in to full tunnel operation if you have a router configured on your network.
### Ad-Hoc Networks
A special kind of public network called an ad-hoc network may be accessed by joining a network ID with the format:
ffSSSSEEEE000000
| | | |
| | | Reserved for future use, must be 0
| | End of port range (hex)
| Start of port range (hex)
Reserved ZeroTier address prefix indicating a controller-less network
Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to desintation ports within the encoded range.
For example `ff00160016000000` is an ad-hoc network allowing only SSH, while `ff0000ffff000000` is an ad-hoc network allowing any UDP or TCP port.
Keep in mind that these networks are public and anyone in the entire world can join them. Care must be taken to avoid exposing vulnerable services or sharing unwanted files or other resources.
### Network Controller (Partial) Rewrite
The network controller has been largely rewritten to use a simple in-filesystem JSON data store in place of SQLite, and it is now included by default in all Windows, Mac, Linux, and BSD builds. This means any desktop or server node running ZeroTier One can now be a controller with no recompilation needed.
If you have data in an old SQLite3 controller we've included a NodeJS script in `controller/migrate-sqlite` to migrate data to the new format. If you don't migrate, members will start getting `NOT_FOUND` when they attempt to query for updates.
## Major Bug Fixes in 1.2.0
* **The Windows HyperV 100% CPU bug is FINALLY DEAD**: This long-running problem turns out to have been an issue with Windows itself, but one we were triggering by placing invalid data into the Windows registry. Microsoft is aware of the issue but we've also fixed the triggering problem on our side. ZeroTier should now co-exist quite well with HyperV and should now be able to be bridged with a HyperV virtual switch.
* **Segmenation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
* **Windows firewall blocks local JSON API**: On some Windows systems the firewall likes to block 127.0.0.1:9993 for mysterious reasons. This is now fixed in the installer via the addition of another firewall exemption rule.
* **UI crash on embedded Windows due to missing fonts**: The MSI installer now ships fonts and will install them if they are not present, so this should be fixed.
## Other Improvements in 1.2.0
* **Improved dead path detection**: ZeroTier is now more aggressive about expiring paths that do not seem to be active. If a path seems marginal it is re-confirmed before re-use.
* **Minor performance improvements**: We've reduced unnecessary memcpy's and made a few other performance improvements in the core.
* **Linux static binaries**: For our official packages (the ones in the download.zerotier.com apt and yum repositories) we now build Linux binaries with static linking. Hopefully this will stop all the bug reports relating to library inconsistencies, as well as allowing our deb packages to run on a wider variety of Debian-based distributions. (There are far too many of these to support officially!) The overhead for this is very small, especially since we built our static versions against musl-libc. Distribution maintainers are of course free to build dynamically linked versions for inclusion into distributions; this only affects our official binaries.
Loaded 20 of 505 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.