0.2.5 - cleaned up multicast propagation algorithm and factored it out into Multicaster.hpp and BloomFilter.hpp

(1) Probable fix for issue #7 and major cleanup of EthernetTap code with consolidation for all unix-like systems and specialization for different flavors only when needed.

(2) Refactor of Buffer<> to make its members private, and Packet to use Buffer's methods exclusively to access them. This improves clarity and means we're no longer lying about Buffer's role in the code's security posture.

(3) Add -fstack-protect to Makefile to bounds check stack variables.

Makefile.linux

@@ -6,7 +6,7 @@ ARCH=$(shell uname -m)
 DEFS=-DZT_ARCH="$(ARCH)" -DZT_OSNAME="linux" -DZT_TRACE
 
 # Uncomment for a release optimized build
-CFLAGS=-Wall -O6 -fno-unroll-loops -pthread $(INCLUDES) -DNDEBUG $(DEFS)
+CFLAGS=-Wall -O3 -fno-unroll-loops -fstack-protector -pthread $(INCLUDES) -DNDEBUG $(DEFS)
 STRIP=strip --strip-all
 
 # Uncomment for a debug build

Makefile.mac

@@ -5,7 +5,7 @@ INCLUDES=-Iext/bin/libcrypto/include -Iext/jsoncpp/include
 DEFS=-DZT_ARCH="x86_combined" -DZT_OSNAME="mac" -DZT_TRACE
 
 # Uncomment for a release optimized universal binary build
-CFLAGS=-arch i386 -arch x86_64 -Wall -O6 -ftree-vectorize -pthread -mmacosx-version-min=10.6 -DNDEBUG $(INCLUDES) $(DEFS)
+CFLAGS=-arch i386 -arch x86_64 -Wall -O3 -ftree-vectorize -fstack-protector -pthread -mmacosx-version-min=10.6 -DNDEBUG $(INCLUDES) $(DEFS)
 STRIP=strip
 
 # Uncomment for a debug build

README.md

@@ -47,8 +47,8 @@ Check out the [blog](http://blog.zerotier.com/) for announcements, in-depth arti
 **Q:** Can you see my traffic? What about other users? Can you sniff the LAN?  
 **A:** No. All unicast (direct computer to computer) traffic is encrypted end-to-end (even if it's being relayed), and the ZeroTier virtual LAN behaves like a LAN with a secure enterprise-grade switch that does not allow unicast conversations to be sniffed. Multicast and broadcast traffic will of course be seen by many recipients, but that's the idea.
 
-**Q:** You say "almost unlimited size." Isn't multicast and broadcast traffic eventually going to be too much? What happens then?  
-**A:** ZeroTier One uses an algorithm called *implicit social switching*. The overall maximum number of recipients for a multicast is limited, so if there are too many listeners to a given multicast address then obviously not everyone will receive every message. So who does? Social switching causes multicasts to propagate in the direction of peers to whom you have recently communicated. As a result, multicasts tend to propagate along lines of association. The people most likely to get your service announcements are those with whom you frequently connect.
+**Q:** What about privacy? Does this hide my location on the network?  
+**A:** ZeroTier is not a connection anonymizer. Other than encryption, it doesn't do anything special to hide your identity or network location. If you want strong privacy protection there are already very advanced tools like [Tor](https://www.torproject.org) for that, and this isn't trying to duplicate their functionality. At the same time, ZT does not do anything special to harm your privacy either. It's not spyware or snoop-ware.
 
 **Q:** Is this designed to replace IP, BGP, IPv6, routers, etc.?  
 **A:** No. Its purpose is to act as a collaboration tool, a VPN alternative, a network mobility tool, a testbed for the development of software that takes advantage of fully open networking, a virtual LAN party for gamers, and so on, but it's not intended (or able) to replace the core of the Internet.
@@ -89,4 +89,7 @@ Check out the [blog](http://blog.zerotier.com/) for announcements, in-depth arti
 
 ----
 
+<a href="http://flattr.com/thing/1611614/ZeroTier-Networks" target="_blank"><img src="http://api.flattr.com/button/flattr-badge-large.png" alt="Flattr this" title="Flattr this" border="0" /></a>  
+[![githalytics.com alpha](https://cruel-carlota.pagodabox.com/59b2cbb9c154bf84bddb4b714402e548 "githalytics.com")](http://githalytics.com/zerotier/ZeroTierOne)
+
 (c)2012-2013 [ZeroTier Networks LLC](https://www.zerotier.com/)

node/BloomFilter.hpp

@@ -0,0 +1,137 @@
+/*
+ * ZeroTier One - Global Peer to Peer Ethernet
+ * Copyright (C) 2012-2013  ZeroTier Networks LLC
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * ZeroTier may be used and distributed under the terms of the GPLv3, which
+ * are available at: http://www.gnu.org/licenses/gpl-3.0.html
+ *
+ * If you would like to embed ZeroTier into a commercial application or
+ * redistribute it in a modified binary form, please contact ZeroTier Networks
+ * LLC. Start here: http://www.zerotier.com/
+ */
+
+#ifndef _ZT_BLOOMFILTER_HPP
+#define _ZT_BLOOMFILTER_HPP
+
+#include <string.h>
+#include "Utils.hpp"
+
+namespace ZeroTier {
+
+/**
+ * A simple bit field bloom filter
+ *
+ * This actually isn't a total filter, in that it does not contain a hashing
+ * algorithm. It's up to the caller to hash/sum the items being remembered
+ * so that the distribution of 'n' is random.
+ *
+ * @tparam B Size in BITS, must be a multiple of 8
+ */
+template<unsigned int B>
+class BloomFilter
+{
+public:
+	/**
+	 * Construct an empty filter
+	 */
+	BloomFilter()
+		throw()
+	{
+		memset(_field,0,sizeof(_field));
+	}
+
+	/**
+	 * Construct from a raw filter
+	 *
+	 * @param b Raw filter bits, must be exactly bytes() in length, or NULL to construct empty
+	 */
+	BloomFilter(const void *b)
+		throw()
+	{
+		if (b)
+			memcpy(_field,b,sizeof(_field));
+		else memset(_field,0,sizeof(_field));
+	}
+
+	/**
+	 * @return Size of filter in bits
+	 */
+	static inline unsigned int bits() throw() { return B; }
+
+	/**
+	 * @return Size of filter in bytes
+	 */
+	static inline unsigned int bytes() throw() { return (B / 8); }
+
+	/**
+	 * @return Pointer to portable array of bytes of bytes() length representing filter
+	 */
+	inline const unsigned char *data() const throw() { return _field; }
+
+	/**
+	 * Clear all bits in filter
+	 */
+	inline void clear()
+		throw()
+	{
+		memset(_field,0,sizeof(_field));
+	}
+
+	/**
+	 * @param n Value to set
+	 * @return True if corresponding bit was already set before this operation
+	 */
+	inline bool set(unsigned int n)
+		throw()
+	{
+		n %= B;
+		unsigned char *const x = _field + (n / 8);
+		const unsigned char m = (1 << (n % 8));
+		bool already = ((*x & m));
+		*x |= m;
+		return already;
+	}
+
+	/**
+	 * @param n Value to test
+	 * @return True if value is present is set
+	 */
+	inline bool contains(unsigned int n)
+		throw()
+	{
+		n %= B;
+		return (_field[n / 8] & (1 << (n % 8)));
+	}
+
+	/**
+	 * Clear a random bit in this bloom filter
+	 */
+	inline void decay()
+		throw()
+	{
+		const unsigned int rn = Utils::randomInt<unsigned int>();
+		_field[(rn >> 3) % (B / 8)] &= ~((unsigned char)(1 << (rn & 7)));
+	}
+
+private:
+	unsigned char _field[B / 8];
+};
+
+} // namespace ZeroTier
+
+#endif

node/Buffer.hpp

@@ -240,6 +240,22 @@ public:
 		_l += sizeof(T);
 	}
 
+	/**
+	 * Append a run of bytes
+	 *
+	 * @param c Character value to append
+	 * @param n Number of times to append
+	 * @throws std::out_of_range Attempt to append beyond capacity
+	 */
+	inline void append(unsigned char c,unsigned int n)
+		throw(std::out_of_range)
+	{
+		if ((_l + n) > C)
+			throw std::out_of_range("Buffer: append beyond capacity");
+		for(unsigned int i=0;i<n;++i)
+			_b[_l++] = (char)c;
+	}
+
 	/**
 	 * Append a C-array of bytes
 	 * 
@@ -388,7 +404,7 @@ public:
 		return !(*this < b);
 	}
 
-protected:
+private:
 	unsigned int _l;
 	char ZT_VAR_MAY_ALIAS _b[C];
 };

node/Constants.hpp

@@ -28,10 +28,55 @@
 #ifndef _ZT_CONSTANTS_HPP
 #define _ZT_CONSTANTS_HPP
 
-// Assume these are little-endian, since we don't support old PPC MACs
-// and all newer Mac or Windows systems are either x86_32, x86_64, or
-// ARM in little-endian mode.
-#if defined(__APPLE__) || defined(_WIN32)
+//
+// This include file also auto-detects and canonicalizes some environment
+// information defines:
+//
+// __LINUX__
+// __APPLE__
+// __UNIX_LIKE__ - any "unix like" OS (BSD, posix, etc.)
+// __WINDOWS__
+//
+// Also makes sure __BYTE_ORDER is defined reasonably.
+//
+
+// Canonicalize Linux... is this necessary? Do it anyway to be defensive.
+#if defined(__linux__) || defined(linux) || defined(__LINUX__) || defined(__linux)
+#ifndef __LINUX__
+#define __LINUX__
+#ifndef __UNIX_LIKE__
+#define __UNIX_LIKE__
+#endif
+#endif
+#endif
+
+// TODO: Android is what? Linux technically, but does it define it?
+
+// OSX and iOS are unix-like OSes far as we're concerned
+#ifdef __APPLE__
+#ifndef __UNIX_LIKE__
+#define __UNIX_LIKE__
+#endif
+#endif
+
+// Linux has endian.h
+#ifdef __LINUX__
+#include <endian.h>
+#endif
+
+#if defined(_WIN32) || defined(_WIN64)
+#ifndef __WINDOWS__
+#define __WINDOWS__
+#endif
+#undef __UNIX_LIKE__
+#define ZT_PATH_SEPARATOR '\\'
+#define ZT_PATH_SEPARATOR_S "\\"
+#define ZT_EOL_S "\r\n"
+#endif
+
+// Assume these are little-endian. PPC is not supported for OSX, and ARM
+// runs in little-endian mode for these OS families.
+#if defined(__APPLE__) || defined(__WINDOWS__)
 #undef __BYTE_ORDER
 #undef __LITTLE_ENDIAN
 #undef __BIG_ENDIAN
@@ -40,33 +85,23 @@
 #define __BYTE_ORDER 1234
 #endif
 
-// Linux has endian.h, which should tell us
-#if defined(__linux__) || defined(linux) || defined(__LINUX__) || defined(__linux)
-#include <endian.h>
-#endif
-
-#ifndef __BYTE_ORDER
-error_no_byte_order_defined
-#endif
-
-#ifndef ZT_OSNAME
-error_no_ZT_OSNAME
-#endif
-
-#ifndef ZT_ARCH
-error_no_ZT_ARCH
-#endif
-
-#ifdef _WIN32
-#define ZT_PATH_SEPARATOR '\\'
-#define ZT_PATH_SEPARATOR_S "\\"
-#define ZT_EOL_S "\r\n"
-#else
+#ifdef __UNIX_LIKE__
 #define ZT_PATH_SEPARATOR '/'
 #define ZT_PATH_SEPARATOR_S "/"
 #define ZT_EOL_S "\n"
 #endif
 
+// Error out if required symbols are missing
+#ifndef __BYTE_ORDER
+error_no_byte_order_defined;
+#endif
+#ifndef ZT_OSNAME
+error_no_ZT_OSNAME_defined;
+#endif
+#ifndef ZT_ARCH
+error_no_ZT_ARCH_defined;
+#endif
+
 /**
  * Length of a ZeroTier address in bytes
  */
@@ -207,6 +242,11 @@ error_no_ZT_ARCH
  */
 #define ZT_MULTICAST_DEDUP_HISTORY_EXPIRE 8000
 
+/**
+ * Number of bits to randomly "decay" in bloom filter per hop
+ */
+#define ZT_MULTICAST_BLOOM_FILTER_DECAY_RATE 2
+
 /**
  * Period between announcements of all multicast 'likes' in ms
  *

node/EthernetTap.hpp

@@ -36,14 +36,13 @@
 #include <set>
 #include <string>
 #include <stdexcept>
-#include "Array.hpp"
-#include "Utils.hpp"
-#include "InetAddress.hpp"
-#include "NonCopyable.hpp"
-#include "MAC.hpp"
 #include "Constants.hpp"
+#include "InetAddress.hpp"
+#include "MAC.hpp"
 #include "Mutex.hpp"
 #include "MulticastGroup.hpp"
+#include "Thread.hpp"
+#include "Buffer.hpp"
 
 namespace ZeroTier {
 
@@ -52,21 +51,40 @@ class RuntimeEnvironment;
 /**
  * System ethernet tap device
  */
-class EthernetTap : NonCopyable
+class EthernetTap : protected Thread
 {
 public:
 	/**
 	 * Construct a new TAP device
 	 *
+	 * Handler arguments: arg,from,to,etherType,data
+	 * 
 	 * @param renv Runtime environment
 	 * @param mac MAC address of device
 	 * @param mtu MTU of device
+	 * @param handler Handler function to be called when data is received from the tap
+	 * @param arg First argument to handler function
 	 * @throws std::runtime_error Unable to allocate device
 	 */
-	EthernetTap(const RuntimeEnvironment *renv,const MAC &mac,unsigned int mtu)
+	EthernetTap(
+		const RuntimeEnvironment *renv,
+		const MAC &mac,
+		unsigned int mtu,
+		void (*handler)(void *,const MAC &,const MAC &,unsigned int,const Buffer<4096> &),
+		void *arg)
 		throw(std::runtime_error);
 
-	~EthernetTap();
+	/**
+	 * Close tap and shut down thread
+	 *
+	 * This may block for a few seconds while thread exits.
+	 */
+	virtual ~EthernetTap();
+
+	/**
+	 * Perform OS dependent actions on network configuration change detection
+	 */
+	void whack();
 
 	/**
 	 * @return MAC address of this interface
@@ -132,31 +150,10 @@ public:
 	 */
 	void put(const MAC &from,const MAC &to,unsigned int etherType,const void *data,unsigned int len);
 
-	/**
-	 * Get the next packet from the interface, blocking if none is available.
-	 *
-	 * @param from Filled with MAC address of source (normally our own)
-	 * @param to Filled with MAC address of destination
-	 * @param etherType Filled with Ethernet frame type
-	 * @param buf Buffer to fill (must have room for MTU bytes)
-	 * @return Number of bytes read or 0 if none
-	 */
-	unsigned int get(MAC &from,MAC &to,unsigned int &etherType,void *buf);
-
 	/**
 	 * @return OS-specific device or connection name
 	 */
-	std::string deviceName();
-
-	/**
-	 * @return True if tap is open
-	 */
-	bool open() const;
-
-	/**
-	 * Close this tap, invalidating the object and causing get() to abort
-	 */
-	void close();
+	std::string deviceName() const;
 
 	/**
 	 * Fill or modify a set to contain multicast groups for this device
@@ -164,11 +161,18 @@ public:
 	 * This populates a set or, if already populated, modifies it to contain
 	 * only multicast groups in which this device is interested.
 	 *
+	 * This should always include the blind wildcard MulticastGroup (MAC of
+	 * ff:ff:ff:ff:ff:ff and 0 ADI field).
+	 *
 	 * @param groups Set to modify in place
 	 * @return True if set was changed since last call
 	 */
 	bool updateMulticastGroups(std::set<MulticastGroup> &groups);
 
+protected:
+	virtual void main()
+		throw();
+
 private:
 	const MAC _mac;
 	const unsigned int _mtu;
@@ -178,20 +182,14 @@ private:
 	std::set<InetAddress> _ips;
 	Mutex _ips_m;
 
-#if defined(__APPLE__) || defined(__linux__) || defined(linux) || defined(__LINUX__) || defined(__linux)
+	void (*_handler)(void *,const MAC &,const MAC &,unsigned int,const Buffer<4096> &);
+	void *_arg;
 
+#ifdef __UNIX_LIKE__
 	char _dev[16];
-	unsigned char *_putBuf;
-	unsigned char *_getBuf;
 	int _fd;
-
-	bool _isReading;
-	pthread_t _isReadingThreadId;
-	Mutex _isReading_m;
-
-#elif defined(_WIN32) /* -------------------------------------------------- */
-
-#endif /* ----------------------------------------------------------------- */
+	int _shutdownSignalPipe[2];
+#endif
 };
 
 } // namespace ZeroTier

node/Filter.cpp

@@ -0,0 +1,241 @@
+/*
+ * ZeroTier One - Global Peer to Peer Ethernet
+ * Copyright (C) 2012-2013  ZeroTier Networks LLC
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * ZeroTier may be used and distributed under the terms of the GPLv3, which
+ * are available at: http://www.gnu.org/licenses/gpl-3.0.html
+ *
+ * If you would like to embed ZeroTier into a commercial application or
+ * redistribute it in a modified binary form, please contact ZeroTier Networks
+ * LLC. Start here: http://www.zerotier.com/
+ */
+
+#include <stdint.h>
+
+#include "RuntimeEnvironment.hpp"
+#include "Logger.hpp"
+#include "Filter.hpp"
+#include "Utils.hpp"
+
+namespace ZeroTier {
+
+bool Filter::Rule::operator()(unsigned int etype,const void *data,unsigned int len) const
+{
+	if ((!_etherType)||(_etherType(etype))) { // ethertype is ANY, or matches
+		// Ethertype determines meaning of protocol and port
+		switch(etype) {
+			default:
+				if ((!_protocol)&&(!_port))
+					return true; // match other ethertypes if protocol and port are ANY, since we don't know what to do with them
+				break;
+
+			case ZT_ETHERTYPE_IPV4:
+				if (len > 20) {
+					if ((!_protocol)||(_protocol(((const uint8_t *)data)[9]))) { // IP protocol
+						if (!_port)
+							return true; // protocol matches or is ANY, port is ANY
+
+						// Don't match on fragments beyond fragment 0. If we've blocked
+						// fragment 0, further fragments will fall on deaf ears anyway.
+						if ((Utils::ntoh(((const uint16_t *)data)[3]) & 0x1fff))
+							return false;
+
+						// Internet header length determines where data begins, in multiples of 32 bits
+						unsigned int ihl = 4 * (((const uint8_t *)data)[0] & 0x0f);
+
+						switch(((const uint8_t *)data)[9]) { // port's meaning depends on IP protocol
+							case ZT_IPPROTO_ICMP:
+								return _port(((const uint8_t *)data)[ihl]); // port = ICMP type
+							case ZT_IPPROTO_TCP:
+							case ZT_IPPROTO_UDP:
+							case ZT_IPPROTO_SCTP:
+							case ZT_IPPROTO_UDPLITE:
+								return _port(((const uint16_t *)data)[(ihl / 2) + 1]); // destination port
+						}
+
+						return false; // no match on port
+					}
+				}
+				break;
+
+			case ZT_ETHERTYPE_IPV6:
+				if (len > 40) {
+					// see: http://stackoverflow.com/questions/17518951/is-the-ipv6-header-really-this-nutty
+					int nextHeader = ((const uint8_t *)data)[6];
+					unsigned int pos = 40;
+					while ((pos < len)&&(nextHeader >= 0)&&(nextHeader != 59)) { // 59 == no next header
+						fprintf(stderr,"[rule] V6: start header parse, header %.2x pos %d\n",nextHeader,pos);
+
+						switch(nextHeader) {
+							case 0: // hop-by-hop options
+							case 60: // destination options
+							case 43: // routing
+							case 135: // mobility (mobile IPv6 options)
+								if (_protocol((unsigned int)nextHeader))
+									return true; // match if our goal was to match any of these
+								nextHeader = ((const uint8_t *)data)[pos];
+								pos += 8 + (8 * ((const uint8_t *)data)[pos + 1]);
+								break;
+							case 44: // fragment
+								if (_protocol(44))
+									return true; // match if our goal was to match fragments
+								nextHeader = ((const uint8_t *)data)[pos];
+								pos += 8;
+								break;
+							case ZT_IPPROTO_AH: // AH
+								return _protocol(ZT_IPPROTO_AH); // true if AH is matched protocol, otherwise false since packet will be IPsec
+							case ZT_IPPROTO_ESP: // ESP
+								return _protocol(ZT_IPPROTO_ESP); // true if ESP is matched protocol, otherwise false since packet will be IPsec
+							case ZT_IPPROTO_ICMPV6:
+								if (_protocol(ZT_IPPROTO_ICMPV6)) { // only match ICMPv6 if specified
+									if ((!_port)||(_port(((const uint8_t *)data)[pos])))
+										return true; // protocol matches, port is ANY or matches ICMP type
+								}
+								break;
+							case ZT_IPPROTO_TCP:
+							case ZT_IPPROTO_UDP:
+							case ZT_IPPROTO_SCTP:
+							case ZT_IPPROTO_UDPLITE:
+								// If we encounter any of these, match if protocol matches or is wildcard as
+								// we'll consider these the "real payload" if present.
+								if ((!_protocol)||(_protocol(nextHeader))) {
+									if ((!_port)||(_port(((const uint16_t *)data)[(pos / 2) + 1])))
+										return true; // protocol matches or is ANY, port is ANY or matches
+								}
+								break;
+						}
+
+						fprintf(stderr,"[rule] V6: end header parse, next header %.2x, new pos %d\n",nextHeader,pos);
+					}
+				}
+				break;
+		}
+	}
+
+	return false;
+}
+
+Filter::Filter(const RuntimeEnvironment *renv) :
+	_r(renv)
+{
+}
+
+Filter::~Filter()
+{
+}
+
+void Filter::add(const Rule &r,const Action &a)
+{
+	Mutex::Lock _l(_chain_m);
+	for(std::vector<Entry>::iterator i(_chain.begin());i!=_chain.end();++i) {
+		if (i->rule == r) {
+			_chain.erase(i);
+			break;
+		}
+	}
+	_chain.push_back(Entry(r,a));
+}
+
+std::string Filter::toString(const char *sep) const
+{
+	char buf[256];
+
+	if (!sep)
+		sep = ",";
+
+	std::string s;
+
+	Mutex::Lock _l(_chain_m);
+	for(std::vector<Entry>::const_iterator i(_chain.begin());i!=_chain.end();++i) {
+		bool first = (i == _chain.begin());
+
+		s.push_back('[');
+
+		if (i->rule.etherType()) {
+			if (i->rule.etherType().magnitude() > 1)
+				sprintf(buf,"%u-%u",i->rule.etherType().start,i->rule.etherType().end);
+			else sprintf(buf,"%u",i->rule.etherType().start);
+			s.append(buf);
+		} else s.push_back('*');
+
+		s.push_back(';');
+
+		if (i->rule.protocol()) {
+			if (i->rule.protocol().magnitude() > 1)
+				sprintf(buf,"%u-%u",i->rule.protocol().start,i->rule.protocol().end);
+			else sprintf(buf,"%u",i->rule.protocol().start);
+			s.append(buf);
+		} else s.push_back('*');
+
+		s.push_back(';');
+
+		if (i->rule.port()) {
+			if (i->rule.port().magnitude() > 1)
+				sprintf(buf,"%u-%u",i->rule.port().start,i->rule.port().end);
+			else sprintf(buf,"%u",i->rule.port().start);
+			s.append(buf);
+		} else s.push_back('*');
+
+		s.append("]:");
+
+		switch(i->action) {
+			case ACTION_DENY:
+				s.append("DENY");
+				break;
+			case ACTION_ALLOW:
+				s.append("ALLOW");
+				break;
+			case ACTION_LOG:
+				s.append("LOG");
+				break;
+		}
+
+		if (!first)
+			s.append(sep);
+	}
+
+	return s;
+}
+
+const char *Filter::etherTypeName(const unsigned int etherType)
+	throw()
+{
+	static char tmp[6];
+	switch(etherType) {
+		case ZT_ETHERTYPE_IPV4:
+			return "IPV4";
+		case ZT_ETHERTYPE_ARP:
+			return "ARP";
+		case ZT_ETHERTYPE_RARP:
+			return "RARP";
+		case ZT_ETHERTYPE_ATALK:
+			return "ATALK";
+		case ZT_ETHERTYPE_AARP:
+			return "AARP";
+		case ZT_ETHERTYPE_IPX_A:
+			return "IPX_A";
+		case ZT_ETHERTYPE_IPX_B:
+			return "IPX_B";
+		case ZT_ETHERTYPE_IPV6:
+			return "IPV6";
+	}
+	sprintf(tmp,"%.4x",etherType);
+	return tmp; // technically not thread safe, but we're only going to see this in debugging if ever
+}
+
+} // namespace ZeroTier

node/Filter.hpp

@@ -0,0 +1,300 @@
+/*
+ * ZeroTier One - Global Peer to Peer Ethernet
+ * Copyright (C) 2012-2013  ZeroTier Networks LLC
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * ZeroTier may be used and distributed under the terms of the GPLv3, which
+ * are available at: http://www.gnu.org/licenses/gpl-3.0.html
+ *
+ * If you would like to embed ZeroTier into a commercial application or
+ * redistribute it in a modified binary form, please contact ZeroTier Networks
+ * LLC. Start here: http://www.zerotier.com/
+ */
+
+#ifndef _ZT_FILTER_HPP
+#define _ZT_FILTER_HPP
+
+#include <stdio.h>
+#include <string.h>
+#include <string>
+#include <vector>
+#include <utility>
+
+#include "Mutex.hpp"
+#include "Range.hpp"
+
+/* Ethernet frame types that might be relevant to us */
+#define ZT_ETHERTYPE_IPV4 0x0800
+#define ZT_ETHERTYPE_ARP 0x0806
+#define ZT_ETHERTYPE_RARP 0x8035
+#define ZT_ETHERTYPE_ATALK 0x809b
+#define ZT_ETHERTYPE_AARP 0x80f3
+#define ZT_ETHERTYPE_IPX_A 0x8137
+#define ZT_ETHERTYPE_IPX_B 0x8138
+#define ZT_ETHERTYPE_IPV6 0x86dd
+
+/* IP protocols we might care about */
+#define ZT_IPPROTO_ICMP 0x01
+#define ZT_IPPROTO_IGMP 0x02
+#define ZT_IPPROTO_TCP 0x06
+#define ZT_IPPROTO_UDP 0x11
+#define ZT_IPPROTO_GRE 0x2f
+#define ZT_IPPROTO_ESP 0x32
+#define ZT_IPPROTO_AH 0x33
+#define ZT_IPPROTO_ICMPV6 0x3a
+#define ZT_IPPROTO_OSPF 0x59
+#define ZT_IPPROTO_IPIP 0x5e
+#define ZT_IPPROTO_IPCOMP 0x6c
+#define ZT_IPPROTO_L2TP 0x73
+#define ZT_IPPROTO_SCTP 0x84
+#define ZT_IPPROTO_FC 0x85
+#define ZT_IPPROTO_UDPLITE 0x88
+#define ZT_IPPROTO_HIP 0x8b
+
+/* IPv4 ICMP types */
+#define ZT_ICMP_ECHO_REPLY 0
+#define ZT_ICMP_DESTINATION_UNREACHABLE 3
+#define ZT_ICMP_SOURCE_QUENCH 4
+#define ZT_ICMP_REDIRECT 5
+#define ZT_ICMP_ALTERNATE_HOST_ADDRESS 6
+#define ZT_ICMP_ECHO_REQUEST 8
+#define ZT_ICMP_ROUTER_ADVERTISEMENT 9
+#define ZT_ICMP_ROUTER_SOLICITATION 10
+#define ZT_ICMP_TIME_EXCEEDED 11
+#define ZT_ICMP_BAD_IP_HEADER 12
+#define ZT_ICMP_TIMESTAMP 13
+#define ZT_ICMP_TIMESTAMP_REPLY 14
+#define ZT_ICMP_INFORMATION_REQUEST 15
+#define ZT_ICMP_INFORMATION_REPLY 16
+#define ZT_ICMP_ADDRESS_MASK_REQUEST 17
+#define ZT_ICMP_ADDRESS_MASK_REPLY 18
+#define ZT_ICMP_TRACEROUTE 30
+#define ZT_ICMP_MOBILE_HOST_REDIRECT 32
+#define ZT_ICMP_MOBILE_REGISTRATION_REQUEST 35
+#define ZT_ICMP_MOBILE_REGISTRATION_REPLY 36
+
+/* IPv6 ICMP types */
+#define ZT_ICMP6_DESTINATION_UNREACHABLE 1
+#define ZT_ICMP6_PACKET_TOO_BIG 2
+#define ZT_ICMP6_TIME_EXCEEDED 3
+#define ZT_ICMP6_PARAMETER_PROBLEM 4
+#define ZT_ICMP6_ECHO_REQUEST 128
+#define ZT_ICMP6_ECHO_REPLY 129
+#define ZT_ICMP6_MULTICAST_LISTENER_QUERY 130
+#define ZT_ICMP6_MULTICAST_LISTENER_REPORT 131
+#define ZT_ICMP6_MULTICAST_LISTENER_DONE 132
+#define ZT_ICMP6_ROUTER_SOLICITATION 133
+#define ZT_ICMP6_ROUTER_ADVERTISEMENT 134
+#define ZT_ICMP6_NEIGHBOR_SOLICITATION 135
+#define ZT_ICMP6_NEIGHBOR_ADVERTISEMENT 136
+#define ZT_ICMP6_REDIRECT_MESSAGE 137
+#define ZT_ICMP6_ROUTER_RENUMBERING 138
+#define ZT_ICMP6_NODE_INFORMATION_QUERY 139
+#define ZT_ICMP6_NODE_INFORMATION_RESPONSE 140
+#define ZT_ICMP6_INV_NEIGHBOR_SOLICITATION 141
+#define ZT_ICMP6_INV_NEIGHBOR_ADVERTISEMENT 142
+#define ZT_ICMP6_MLDV2 143
+#define ZT_ICMP6_HOME_AGENT_ADDRESS_DISCOVERY_REQUEST 144
+#define ZT_ICMP6_HOME_AGENT_ADDRESS_DISCOVERY_REPLY 145
+#define ZT_ICMP6_MOBILE_PREFIX_SOLICITATION 146
+#define ZT_ICMP6_MOBILE_PREFIX_ADVERTISEMENT 147
+#define ZT_ICMP6_CERTIFICATION_PATH_SOLICITATION 148
+#define ZT_ICMP6_CERTIFICATION_PATH_ADVERTISEMENT 149
+#define ZT_ICMP6_MULTICAST_ROUTER_ADVERTISEMENT 151
+#define ZT_ICMP6_MULTICAST_ROUTER_SOLICITATION 152
+#define ZT_ICMP6_MULTICAST_ROUTER_TERMINATION 153
+#define ZT_ICMP6_RPL_CONTROL_MESSAGE 155
+
+namespace ZeroTier {
+
+class RuntimeEnvironment;
+
+/**
+ * A simple Ethernet frame level filter supporting basic IP port DENY
+ */
+class Filter
+{
+public:
+	/**
+	 * A filter rule
+	 *
+	 * This behaves as an immutable value object.
+	 */
+	class Rule
+	{
+	public:
+		Rule()
+			throw() :
+			_etherType(),
+			_protocol(),
+			_port()
+		{
+		}
+
+		/**
+		 * Construct a new rule
+		 *
+		 * @param etype Ethernet type or empty range for ANY
+		 * @param prot Protocol or empty range for ANY (meaning depends on ethertype, e.g. IP protocol numbers)
+		 * @param prt Port or empty range for ANY (only applies to some protocols)
+		 */
+		Rule(const Range<unsigned int> &etype,const Range<unsigned int> &prot,const Range<unsigned int> &prt)
+			throw() :
+			_etherType(etype),
+			_protocol(prot),
+			_port(prt)
+		{
+		}
+
+		inline const Range<unsigned int> &etherType() const throw() { return _etherType; }
+		inline const Range<unsigned int> &protocol() const throw() { return _protocol; }
+		inline const Range<unsigned int> &port() const throw() { return _port; }
+
+		/**
+		 * Test this rule against a frame
+		 *
+		 * @param etype Type of ethernet frame
+		 * @param data Ethernet frame data
+		 * @param len Length of ethernet frame
+		 * @return True if rule matches
+		 */
+		bool operator()(unsigned int etype,const void *data,unsigned int len) const;
+
+		inline bool operator==(const Rule &r) const throw() { return ((_etherType == r._etherType)&&(_protocol == r._protocol)&&(_port == r._port)); }
+		inline bool operator!=(const Rule &r) const throw() { return !(*this == r); }
+		inline bool operator<(const Rule &r) const
+			throw()
+		{
+			if (_etherType < r._etherType)
+				return true;
+			else if (_etherType == r._etherType) {
+				if (_protocol < r._protocol)
+					return true;
+				else if (_protocol == r._protocol) {
+					if (_port < r._port)
+						return true;
+				}
+			}
+			return false;
+		}
+		inline bool operator>(const Rule &r) const throw() { return (r < *this); }
+		inline bool operator<=(const Rule &r) const throw() { return !(r < *this); }
+		inline bool operator>=(const Rule &r) const throw() { return !(*this < r); }
+
+	private:
+		Range<unsigned int> _etherType;
+		Range<unsigned int> _protocol;
+		Range<unsigned int> _port;
+	};
+
+	/**
+	 * Action if a rule matches
+	 */
+	enum Action
+	{
+		ACTION_DENY = 0,
+		ACTION_ALLOW = 1,
+		ACTION_LOG = 2
+	};
+
+	/**
+	 * Entry in filter chain
+	 */
+	struct Entry
+	{
+		Entry() {}
+		Entry(const Rule &r,const Action &a) :
+			rule(r),
+			action(a)
+		{
+		}
+
+		Rule rule;
+		Action action;
+	};
+
+	Filter(const RuntimeEnvironment *renv);
+	~Filter();
+
+	/**
+	 * Remove all filter entries
+	 */
+	inline void clear()
+	{
+		Mutex::Lock _l(_chain_m);
+		_chain.clear();
+	}
+
+	/**
+	 * Append a rule/action pair to this chain
+	 *
+	 * If an identical rule already exists it is removed and a new entry is
+	 * added to the end with the new action. (Two identical rules with the
+	 * same action wouldn't make sense.)
+	 *
+	 * @param r Rule to add
+	 * @param a Action if rule matches
+	 */
+	void add(const Rule &r,const Action &a);
+
+	/**
+	 * @return Number of rules in filter chain
+	 */
+	inline unsigned int length() const
+		throw()
+	{
+		Mutex::Lock _l(_chain_m);
+		return _chain.size();
+	}
+
+	/**
+	 * @return Entry in filter chain or null entry if out of bounds
+	 */
+	inline Entry operator[](const unsigned int i) const
+		throw()
+	{
+		Mutex::Lock _l(_chain_m);
+		if (i < _chain.size())
+			return _chain[i];
+		return Entry();
+	}
+
+	/**
+	 * Get a string representation of this filter
+	 *
+	 * @param sep Separator between filter rules, or NULL for comma (default)
+	 * @return Human-readable string
+	 */
+	std::string toString(const char *sep = (const char *)0) const;
+
+	/**
+	 * @param etherType Ethernet type ID
+	 * @return Name of Ethernet protocol (e.g. ARP, IPV4)
+	 */
+	static const char *etherTypeName(const unsigned int etherType)
+		throw();
+
+private:
+	const RuntimeEnvironment *_r;
+
+	std::vector<Entry> _chain;
+	Mutex _chain_m;
+};
+
+} // namespace ZeroTier
+
+#endif

node/Multicaster.hpp

@@ -0,0 +1,274 @@
+/*
+ * ZeroTier One - Global Peer to Peer Ethernet
+ * Copyright (C) 2012-2013  ZeroTier Networks LLC
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * ZeroTier may be used and distributed under the terms of the GPLv3, which
+ * are available at: http://www.gnu.org/licenses/gpl-3.0.html
+ *
+ * If you would like to embed ZeroTier into a commercial application or
+ * redistribute it in a modified binary form, please contact ZeroTier Networks
+ * LLC. Start here: http://www.zerotier.com/
+ */
+
+#ifndef _ZT_MULTICASTER_HPP
+#define _ZT_MULTICASTER_HPP
+
+#include <stdint.h>
+#include <string.h>
+
+#include <utility>
+#include <algorithm>
+#include <map>
+#include <set>
+#include <vector>
+
+#include "Constants.hpp"
+#include "Buffer.hpp"
+#include "Packet.hpp"
+#include "MulticastGroup.hpp"
+#include "Utils.hpp"
+#include "MAC.hpp"
+#include "Address.hpp"
+#include "SharedPtr.hpp"
+#include "BloomFilter.hpp"
+
+// Maximum sample size to pick during choice of multicast propagation peers
+#define ZT_MULTICAST_PICK_MAX_SAMPLE_SIZE 64
+
+namespace ZeroTier {
+
+/**
+ * Multicast propagation engine
+ *
+ * This is written as a generic class so that it can be mocked and tested
+ * in simulation. It also always takes 'now' as an argument, permitting
+ * running in simulated time.
+ */
+class Multicaster
+{
+public:
+	/**
+	 * 256-bit simple bloom filter included with multicast frame packets
+	 */
+	typedef BloomFilter<ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE_BITS> MulticastBloomFilter;
+
+	Multicaster()
+	{
+		memset(_multicastHistory,0,sizeof(_multicastHistory));
+	}
+
+	/**
+	 * Update the most recent LIKE time for an address in a given multicast group on a given network
+	 *
+	 * @param nwid Network ID
+	 * @param mg Multicast group
+	 * @param addr Address that likes group on given network
+	 * @param now Current timestamp
+	 */
+	inline void likesMulticastGroup(const uint64_t nwid,const MulticastGroup &mg,const Address &addr,const uint64_t now)
+	{
+		_multicastMemberships[MulticastChannel(nwid,mg)][addr] = now;
+	}
+
+	/**
+	 * Check multicast history to see if this is a duplicate, and add/update entry
+	 *
+	 * @param from Ultimate sending MAC address
+	 * @param to Destination multicast group
+	 * @param payload Multicast packet payload
+	 * @param len Length of packet
+	 * @param nwid Network ID
+	 * @param now Current time
+	 * @return True if this appears to be a duplicate to within history expiration time
+	 */
+	inline bool checkAndUpdateMulticastHistory(
+		const MAC &from,
+		const MulticastGroup &to,
+		const void *payload,
+		unsigned int len,
+		const uint64_t nwid,
+		const uint64_t now)
+		throw()
+	{
+		// Note: CRCs aren't transmitted over the network, so portability and
+		// byte order don't matter. This calculation can be changed. We just
+		// want a unique code.
+		uint64_t crc = Utils::crc64(0,from.data,6);
+		crc = Utils::crc64(crc,to.mac().data,6);
+		crc ^= (uint64_t)to.adi();
+		crc = Utils::crc64(crc,payload,len);
+		crc ^= nwid; // also include network ID in CRC
+
+		// Replace existing entry or pick one to replace with new entry
+		uint64_t earliest = 0xffffffffffffffffULL;
+		unsigned long earliestIdx = 0;
+		for(unsigned int i=0;i<ZT_MULTICAST_DEDUP_HISTORY_LENGTH;++i) {
+			if (_multicastHistory[i][0] == crc) {
+				uint64_t then = _multicastHistory[i][1];
+				_multicastHistory[i][1] = now;
+				return ((now - then) < ZT_MULTICAST_DEDUP_HISTORY_EXPIRE);
+			} else if (_multicastHistory[i][1] < earliest) {
+				earliest = _multicastHistory[i][1];
+				earliestIdx = i;
+			}
+		}
+
+		_multicastHistory[earliestIdx][0] = crc; // replace oldest entry
+		_multicastHistory[earliestIdx][1] = now;
+
+		return false;
+	}
+
+	/**
+	 * Choose peers to send a propagating multicast to
+	 *
+	 * @param topology Topology object or mock thereof
+	 * @param nwid Network ID
+	 * @param mg Multicast group
+	 * @param upstream Address from which message originated, or null (0) address if none
+	 * @param bf Bloom filter, updated in place with sums of addresses in chosen peers and/or decay
+	 * @param max Maximum number of peers to pick
+	 * @param peers Array of objects of type P to fill with up to [max] peers
+	 * @param now Current timestamp
+	 * @return Number of peers actually stored in peers array
+	 * @tparam T Type of topology, which is Topology in running code or a mock in simulation
+	 * @tparam P Type of peers, which is SharedPtr<Peer> in running code or a mock in simulation (mock must behave like a pointer type)
+	 */
+	template<typename T,typename P>
+	inline unsigned int pickNextPropagationPeers(
+		T &topology,
+		uint64_t nwid,
+		const MulticastGroup &mg,
+		const Address &upstream,
+		MulticastBloomFilter &bf,
+		unsigned int max,
+		P *peers,
+		uint64_t now)
+	{
+		P toConsider[ZT_MULTICAST_PICK_MAX_SAMPLE_SIZE];
+		unsigned int sampleSize = 0;
+
+		{
+			Mutex::Lock _l(_multicastMemberships_m);
+
+			// Sample a random subset of peers that we know have LIKEd this multicast
+			// group on this network.
+			std::map< MulticastChannel,std::map<Address,uint64_t> >::iterator channelMembers(_multicastMemberships.find(MulticastChannel(nwid,mg)));
+			if ((channelMembers != _multicastMemberships.end())&&(!channelMembers->second.empty())) {
+				unsigned long numEntriesPermittedToSkip = (channelMembers->second.size() > ZT_MULTICAST_PICK_MAX_SAMPLE_SIZE) ? (unsigned long)(channelMembers->second.size() - ZT_MULTICAST_PICK_MAX_SAMPLE_SIZE) : (unsigned long)0;
+				double skipWhatFraction = (double)numEntriesPermittedToSkip / (double)channelMembers->second.size();
+
+				std::map<Address,uint64_t>::iterator channelMemberEntry(channelMembers->second.begin());
+
+				while (channelMemberEntry != channelMembers->second.end()) {
+					// Auto-clean the channel members map if their LIKEs are expired. This will
+					// technically skew the random distribution of chosen members just a little, but
+					// it's unlikely that enough will expire in any single pick to make much of a
+					// difference overall.
+					if ((now - channelMemberEntry->second) > ZT_MULTICAST_LIKE_EXPIRE) {
+						channelMembers->second.erase(channelMemberEntry++);
+						continue;
+					}
+
+					// Skip some fraction of entries so that our sampling will be randomly distributed,
+					// since there is no other good way to sample randomly from a map.
+					if (numEntriesPermittedToSkip) {
+						double skipThis = (double)(Utils::randomInt<uint32_t>()) / 4294967296.0;
+						if (skipThis <= skipWhatFraction) {
+							--numEntriesPermittedToSkip;
+							++channelMemberEntry;
+							continue;
+						}
+					}
+
+					// If it's not expired and it's from our random sample, add it to the set of peers
+					// to consider.
+					P peer = topology.getPeer(channelMemberEntry->first);
+					if (peer) {
+						toConsider[sampleSize++] = peer;
+						if (sampleSize >= ZT_MULTICAST_PICK_MAX_SAMPLE_SIZE)
+							break; // abort if we have enough candidates
+					}
+					++channelMemberEntry;
+				}
+
+				// Auto-clean: erase whole map if there are no more LIKEs for this channel
+				if (channelMembers->second.empty())
+					_multicastMemberships.erase(channelMembers);
+			}
+		}
+
+		// Sort in descending order of most recent direct unicast frame, picking
+		// peers with whom we have recently communicated. This is "implicit social
+		// switching."
+		std::sort(&(toConsider[0]),&(toConsider[sampleSize]),PeerPropagationPrioritySortOrder<P>());
+
+		// Decay a few random bits in bloom filter to probabilistically eliminate
+		// false positives as we go. The odds of decaying an already-set bit
+		// increases as the bloom filter saturates, so in the early hops of
+		// propagation this likely won't have any effect.
+		for(unsigned int i=0;i<ZT_MULTICAST_BLOOM_FILTER_DECAY_RATE;++i)
+			bf.decay();
+
+		// Pick peers not in the bloom filter, setting bloom filter bits accordingly to
+		// remember and pass on these picks.
+		unsigned int picked = 0;
+		for(unsigned int i=0;((i<sampleSize)&&(picked < max));++i) {
+			if (!bf.set(toConsider[i]->address().sum()))
+				peers[picked++] = toConsider[i];
+		}
+
+		// Add a supernode if there's nowhere else to go. Supernodes know of all multicast
+		// LIKEs and so can act to bridge sparse multicast groups. We do not remember them
+		// in the bloom filter, since such bridging may very well need to happen more than
+		// once.
+		if (!picked) {
+			P peer = topology.getBestSupernode();
+			if (peer)
+				peers[picked++] = peer;
+		}
+
+		return picked;
+	}
+
+private:
+	// Sort order for chosen propagation peers
+	template<typename P>
+	struct PeerPropagationPrioritySortOrder
+	{
+		inline bool operator()(const P &p1,const P &p2) const
+		{
+			return (p1->lastUnicastFrame() >= p2->lastUnicastFrame());
+		}
+	};
+
+	// [0] - CRC, [1] - timestamp
+	uint64_t _multicastHistory[ZT_MULTICAST_DEDUP_HISTORY_LENGTH][2];
+
+	// A multicast channel, essentially a pub/sub channel. It consists of a
+	// network ID and a multicast group within that network.
+	typedef std::pair<uint64_t,MulticastGroup> MulticastChannel;
+
+	// Address and time of last LIKE, by network ID and multicast group
+	std::map< MulticastChannel,std::map<Address,uint64_t> > _multicastMemberships;
+	Mutex _multicastMemberships_m;
+};
+
+} // namespace ZeroTier
+
+#endif

node/Network.cpp

@@ -32,49 +32,29 @@ namespace ZeroTier {
 
 Network::Network(const RuntimeEnvironment *renv,uint64_t id)
 	throw(std::runtime_error) :
-	Thread(),
 	_r(renv),
 	_id(id),
-	_tap(renv,renv->identity.address().toMAC(),ZT_IF_MTU),
+	_tap(renv,renv->identity.address().toMAC(),ZT_IF_MTU,&_CBhandleTapData,this),
 	_members(),
 	_open(false),
 	_lock()
 {
-	TRACE("new network %llu created, TAP device: %s",id,_tap.deviceName().c_str());
-	start();
 }
 
 Network::~Network()
 {
-	_tap.close();
-	join();
-	TRACE("network %llu (%s) closed",_id,_tap.deviceName().c_str());
 }
 
-void Network::main()
-	throw()
+void Network::_CBhandleTapData(void *arg,const MAC &from,const MAC &to,unsigned int etherType,const Buffer<4096> &data)
 {
-	Buffer<4096> buf;
-	MAC from,to;
-	unsigned int etherType = 0;
-
-	while (_tap.open()) {
-		unsigned int len = _tap.get(from,to,etherType,buf.data());
-		if (len) {
-			buf.setSize(len);
-			try {
-				if (!*__refCount)
-					break; // sanity check
-				_r->sw->onLocalEthernet(SharedPtr<Network>(this),from,to,etherType,buf);
-			} catch (std::exception &exc) {
-				TRACE("unexpected exception handling local packet: %s",exc.what());
-			} catch ( ... ) {
-				TRACE("unexpected exception handling local packet");
-			}
-		} else break;
+	const RuntimeEnvironment *_r = ((Network *)arg)->_r;
+	try {
+		_r->sw->onLocalEthernet(SharedPtr<Network>((Network *)arg),from,to,etherType,data);
+	} catch (std::exception &exc) {
+		TRACE("unexpected exception handling local packet: %s",exc.what());
+	} catch ( ... ) {
+		TRACE("unexpected exception handling local packet");
 	}
-
-	TRACE("network %llu thread terminating",_id);
 }
 
 } // namespace ZeroTier

node/Network.hpp

@@ -40,8 +40,9 @@
 #include "SharedPtr.hpp"
 #include "AtomicCounter.hpp"
 #include "RuntimeEnvironment.hpp"
-#include "Thread.hpp"
 #include "MulticastGroup.hpp"
+#include "NonCopyable.hpp"
+#include "MAC.hpp"
 
 namespace ZeroTier {
 
@@ -50,17 +51,17 @@ class NodeConfig;
 /**
  * Local network endpoint
  */
-class Network : protected Thread
+class Network : NonCopyable
 {
 	friend class SharedPtr<Network>;
 	friend class NodeConfig;
 
 private:
-	virtual ~Network();
-
 	Network(const RuntimeEnvironment *renv,uint64_t id)
 		throw(std::runtime_error);
 
+	~Network();
+
 public:
 	/**
 	 * @return Network ID
@@ -111,6 +112,19 @@ public:
 		return ((_open)||(_members.count(addr) > 0));
 	}
 
+	/**
+	 * Shortcut to check open(), whether MAC is ZeroTier, then isMember()
+	 *
+	 * @param mac MAC address to check
+	 * @return True if MAC is allowed
+	 */
+	inline bool isAllowed(const MAC &mac) const
+		throw()
+	{
+		Mutex::Lock _l(_lock);
+		return ((_open)||((mac.isZeroTier())&&(_members.count(Address(mac)) > 0)));
+	}
+
 	/**
 	 * @return True if network is open (no membership required)
 	 */
@@ -141,11 +155,9 @@ public:
 		return _multicastGroups;
 	}
 
-protected:
-	virtual void main()
-		throw();
-
 private:
+	static void _CBhandleTapData(void *arg,const MAC &from,const MAC &to,unsigned int etherType,const Buffer<4096> &data);
+
 	const RuntimeEnvironment *_r;
 	uint64_t _id;
 	EthernetTap _tap;

node/Node.cpp

@@ -273,11 +273,12 @@ Node::ReasonForTermination Node::run()
 				lastNetworkFingerprintCheck = now;
 				uint64_t fp = _r->sysEnv->getNetworkConfigurationFingerprint();
 				if (fp != networkConfigurationFingerprint) {
-					LOG("netconf fingerprint change: %.16llx != %.16llx, pinging all peers",networkConfigurationFingerprint,fp);
+					LOG("netconf fingerprint change: %.16llx != %.16llx, resyncing with network",networkConfigurationFingerprint,fp);
 					networkConfigurationFingerprint = fp;
 					pingAll = true;
 					lastAutoconfigureCheck = 0; // check autoconf after network config change
 					lastMulticastCheck = 0; // check multicast group membership after network config change
+					_r->nc->whackAllTaps(); // call whack() on all tap devices
 				}
 			}
 

node/NodeConfig.hpp

@@ -78,6 +78,17 @@ public:
 		return nwlist;
 	}
 
+	/**
+	 * Call whack() on all networks' tap devices
+	 */
+	inline void whackAllTaps()
+	{
+		std::vector< SharedPtr<Network> > nwlist;
+		Mutex::Lock _l(_networks_m);
+		for(std::map< uint64_t,SharedPtr<Network> >::const_iterator n(_networks.begin());n!=_networks.end();++n)
+			n->second->tap().whack();
+	}
+
 	/**
 	 * @param nwid Network ID
 	 * @return True if this network exists

node/Packet.hpp

@@ -122,7 +122,9 @@
  */
 #define ZT_PROTO_MIN_FRAGMENT_LENGTH ZT_PACKET_FRAGMENT_IDX_PAYLOAD
 
+// Size of bloom filter used in multicast propagation
 #define ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE 32
+#define ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE_BITS 256
 
 // Field incides for parsing verbs
 #define ZT_PROTO_VERB_HELLO_IDX_PROTOCOL_VERSION (ZT_PACKET_IDX_PAYLOAD)
@@ -263,13 +265,13 @@ public:
 			setSize(fragLen + ZT_PROTO_MIN_FRAGMENT_LENGTH);
 
 			// NOTE: this copies both the IV/packet ID and the destination address.
-			memcpy(_b + ZT_PACKET_FRAGMENT_IDX_PACKET_ID,p.data() + ZT_PACKET_IDX_IV,13);
+			memcpy(field(ZT_PACKET_FRAGMENT_IDX_PACKET_ID,13),p.data() + ZT_PACKET_IDX_IV,13);
 
-			_b[ZT_PACKET_FRAGMENT_IDX_FRAGMENT_INDICATOR] = ZT_PACKET_FRAGMENT_INDICATOR;
-			_b[ZT_PACKET_FRAGMENT_IDX_FRAGMENT_NO] = (char)(((fragTotal & 0xf) << 4) | (fragNo & 0xf));
-			_b[ZT_PACKET_FRAGMENT_IDX_HOPS] = 0;
+			(*this)[ZT_PACKET_FRAGMENT_IDX_FRAGMENT_INDICATOR] = ZT_PACKET_FRAGMENT_INDICATOR;
+			(*this)[ZT_PACKET_FRAGMENT_IDX_FRAGMENT_NO] = (char)(((fragTotal & 0xf) << 4) | (fragNo & 0xf));
+			(*this)[ZT_PACKET_FRAGMENT_IDX_HOPS] = 0;
 
-			memcpy(_b + ZT_PACKET_FRAGMENT_IDX_PAYLOAD,p.data() + fragStart,fragLen);
+			memcpy(field(ZT_PACKET_FRAGMENT_IDX_PAYLOAD,fragLen),p.data() + fragStart,fragLen);
 		}
 
 		/**
@@ -277,12 +279,12 @@ public:
 		 * 
 		 * @return Destination ZT address
 		 */
-		inline Address destination() const { return Address(_b + ZT_PACKET_FRAGMENT_IDX_DEST); }
+		inline Address destination() const { return Address(field(ZT_PACKET_FRAGMENT_IDX_DEST,ZT_ADDRESS_LENGTH)); }
 
 		/**
 		 * @return True if fragment is of a valid length
 		 */
-		inline bool lengthValid() const { return (_l >= ZT_PACKET_FRAGMENT_IDX_PAYLOAD); }
+		inline bool lengthValid() const { return (size() >= ZT_PACKET_FRAGMENT_IDX_PAYLOAD); }
 
 		/**
 		 * @return ID of packet this is a fragment of
@@ -292,36 +294,38 @@ public:
 		/**
 		 * @return Total number of fragments in packet
 		 */
-		inline unsigned int totalFragments() const { return (((unsigned int)_b[ZT_PACKET_FRAGMENT_IDX_FRAGMENT_NO] >> 4) & 0xf); }
+		inline unsigned int totalFragments() const { return (((unsigned int)((*this)[ZT_PACKET_FRAGMENT_IDX_FRAGMENT_NO]) >> 4) & 0xf); }
 
 		/**
 		 * @return Fragment number of this fragment
 		 */
-		inline unsigned int fragmentNumber() const { return ((unsigned int)_b[ZT_PACKET_FRAGMENT_IDX_FRAGMENT_NO] & 0xf); }
+		inline unsigned int fragmentNumber() const { return ((unsigned int)((*this)[ZT_PACKET_FRAGMENT_IDX_FRAGMENT_NO]) & 0xf); }
 
 		/**
 		 * @return Fragment ZT hop count
 		 */
-		inline unsigned int hops() const { return (unsigned int)_b[ZT_PACKET_FRAGMENT_IDX_HOPS]; }
+		inline unsigned int hops() const { return (unsigned int)((*this)[ZT_PACKET_FRAGMENT_IDX_HOPS]); }
 
 		/**
 		 * Increment this packet's hop count
 		 */
 		inline void incrementHops()
 		{
-			_b[ZT_PACKET_FRAGMENT_IDX_HOPS] = (_b[ZT_PACKET_FRAGMENT_IDX_HOPS] + 1) & ZT_PROTO_MAX_HOPS;
+			(*this)[ZT_PACKET_FRAGMENT_IDX_HOPS] = (((*this)[ZT_PACKET_FRAGMENT_IDX_HOPS]) + 1) & ZT_PROTO_MAX_HOPS;
 		}
 
-		/**
-		 * @return Fragment payload
-		 */
-		inline unsigned char *payload() { return (unsigned char *)(_b + ZT_PACKET_FRAGMENT_IDX_PAYLOAD); }
-		inline const unsigned char *payload() const { return (const unsigned char *)(_b + ZT_PACKET_FRAGMENT_IDX_PAYLOAD); }
-
 		/**
 		 * @return Length of payload in bytes
 		 */
-		inline unsigned int payloadLength() const { return ((_l > ZT_PACKET_FRAGMENT_IDX_PAYLOAD) ? (_l - ZT_PACKET_FRAGMENT_IDX_PAYLOAD) : 0); }
+		inline unsigned int payloadLength() const { return ((size() > ZT_PACKET_FRAGMENT_IDX_PAYLOAD) ? (size() - ZT_PACKET_FRAGMENT_IDX_PAYLOAD) : 0); }
+
+		/**
+		 * @return Raw packet payload
+		 */
+		inline const unsigned char *payload() const
+		{
+			return field(ZT_PACKET_FRAGMENT_IDX_PAYLOAD,size() - ZT_PACKET_FRAGMENT_IDX_PAYLOAD);
+		}
 	};
 
 	/**
@@ -417,7 +421,7 @@ public:
 		 *   <[4] multicast additional distinguishing information (ADI)>
 		 *   <[32] multicast propagation bloom filter>
 		 *   <[1] 8-bit strict propagation hop count>
-		 *   <[2] 16-bit average peer multicast bandwidth load>
+		 *   <[2] reserved, must be 0>
 		 *   <[6] source Ethernet address>
 		 *   <[2] 16-bit ethertype>
 		 *   <[...] ethernet payload>
@@ -495,8 +499,8 @@ public:
 	Packet() :
 		Buffer<ZT_PROTO_MAX_PACKET_LENGTH>(ZT_PROTO_MIN_PACKET_LENGTH)
 	{
-		Utils::getSecureRandom(_b + ZT_PACKET_IDX_IV,8);
-		_b[ZT_PACKET_IDX_FLAGS] = 0; // zero flags and hops
+		Utils::getSecureRandom(field(ZT_PACKET_IDX_IV,8),8);
+		(*this)[ZT_PACKET_IDX_FLAGS] = 0; // zero flags and hops
 	}
 
 	/**
@@ -509,10 +513,10 @@ public:
 	Packet(const Address &dest,const Address &source,const Verb v) :
 		Buffer<ZT_PROTO_MAX_PACKET_LENGTH>(ZT_PROTO_MIN_PACKET_LENGTH)
 	{
-		Utils::getSecureRandom(_b + ZT_PACKET_IDX_IV,8);
+		Utils::getSecureRandom(field(ZT_PACKET_IDX_IV,8),8);
 		setDestination(dest);
 		setSource(source);
-		_b[ZT_PACKET_IDX_FLAGS] = 0; // zero flags and hops
+		(*this)[ZT_PACKET_IDX_FLAGS] = 0; // zero flags and hops
 		setVerb(v);
 	}
 
@@ -526,10 +530,10 @@ public:
 	inline void reset(const Address &dest,const Address &source,const Verb v)
 	{
 		setSize(ZT_PROTO_MIN_PACKET_LENGTH);
-		Utils::getSecureRandom(_b + ZT_PACKET_IDX_IV,8);
+		Utils::getSecureRandom(field(ZT_PACKET_IDX_IV,8),8);
 		setDestination(dest);
 		setSource(source);
-		_b[ZT_PACKET_IDX_FLAGS] = 0; // zero flags and hops
+		(*this)[ZT_PACKET_IDX_FLAGS] = 0; // zero flags and hops
 		setVerb(v);
 	}
 
@@ -540,8 +544,9 @@ public:
 	 */
 	inline void setDestination(const Address &dest)
 	{
+		unsigned char *d = field(ZT_PACKET_IDX_DEST,ZT_ADDRESS_LENGTH);
 		for(unsigned int i=0;i<ZT_ADDRESS_LENGTH;++i)
-			_b[i + ZT_PACKET_IDX_DEST] = dest[i];
+			d[i] = dest[i];
 	}
 
 	/**
@@ -551,8 +556,9 @@ public:
 	 */
 	inline void setSource(const Address &source)
 	{
+		unsigned char *s = field(ZT_PACKET_IDX_SOURCE,ZT_ADDRESS_LENGTH);
 		for(unsigned int i=0;i<ZT_ADDRESS_LENGTH;++i)
-			_b[i + ZT_PACKET_IDX_SOURCE] = source[i];
+			s[i] = source[i];
 	}
 
 	/**
@@ -560,29 +566,29 @@ public:
 	 * 
 	 * @return Destination ZT address
 	 */
-	inline Address destination() const { return Address(_b + ZT_PACKET_IDX_DEST); }
+	inline Address destination() const { return Address(field(ZT_PACKET_IDX_DEST,ZT_ADDRESS_LENGTH)); }
 
 	/**
 	 * Get this packet's source
 	 * 
 	 * @return Source ZT address
 	 */
-	inline Address source() const { return Address(_b + ZT_PACKET_IDX_SOURCE); }
+	inline Address source() const { return Address(field(ZT_PACKET_IDX_SOURCE,ZT_ADDRESS_LENGTH)); }
 
 	/**
 	 * @return True if packet is of valid length
 	 */
-	inline bool lengthValid() const { return (_l >= ZT_PROTO_MIN_PACKET_LENGTH); }
+	inline bool lengthValid() const { return (size() >= ZT_PROTO_MIN_PACKET_LENGTH); }
 
 	/**
 	 * @return True if packet is encrypted
 	 */
-	inline bool encrypted() const { return (((unsigned char)_b[ZT_PACKET_IDX_FLAGS] & ZT_PROTO_FLAG_ENCRYPTED)); }
+	inline bool encrypted() const { return (((unsigned char)(*this)[ZT_PACKET_IDX_FLAGS] & ZT_PROTO_FLAG_ENCRYPTED)); }
 
 	/**
 	 * @return True if packet is fragmented (expect fragments)
 	 */
-	inline bool fragmented() const { return (((unsigned char)_b[ZT_PACKET_IDX_FLAGS] & ZT_PROTO_FLAG_FRAGMENTED)); }
+	inline bool fragmented() const { return (((unsigned char)(*this)[ZT_PACKET_IDX_FLAGS] & ZT_PROTO_FLAG_FRAGMENTED)); }
 
 	/**
 	 * Set this packet's fragmented flag
@@ -592,26 +598,26 @@ public:
 	inline void setFragmented(bool f)
 	{
 		if (f)
-			_b[ZT_PACKET_IDX_FLAGS] |= (char)ZT_PROTO_FLAG_FRAGMENTED;
-		else _b[ZT_PACKET_IDX_FLAGS] &= (char)(~ZT_PROTO_FLAG_FRAGMENTED);
+			(*this)[ZT_PACKET_IDX_FLAGS] |= (char)ZT_PROTO_FLAG_FRAGMENTED;
+		else (*this)[ZT_PACKET_IDX_FLAGS] &= (char)(~ZT_PROTO_FLAG_FRAGMENTED);
 	}
 
 	/**
 	 * @return True if compressed (result only valid if unencrypted)
 	 */
-	inline bool compressed() const { return (((unsigned char)_b[ZT_PACKET_IDX_VERB] & ZT_PROTO_VERB_FLAG_COMPRESSED)); }
+	inline bool compressed() const { return (((unsigned char)(*this)[ZT_PACKET_IDX_VERB] & ZT_PROTO_VERB_FLAG_COMPRESSED)); }
 
 	/**
 	 * @return ZeroTier forwarding hops (0 to 7)
 	 */
-	inline unsigned int hops() const { return ((unsigned int)_b[ZT_PACKET_IDX_FLAGS] & 0x07); }
+	inline unsigned int hops() const { return ((unsigned int)(*this)[ZT_PACKET_IDX_FLAGS] & 0x07); }
 
 	/**
 	 * Increment this packet's hop count
 	 */
 	inline void incrementHops()
 	{
-		_b[ZT_PACKET_IDX_FLAGS] = (char)((unsigned char)_b[ZT_PACKET_IDX_FLAGS] & 0xf8) | (((unsigned char)_b[ZT_PACKET_IDX_FLAGS] + 1) & 0x07);
+		(*this)[ZT_PACKET_IDX_FLAGS] = (char)((unsigned char)(*this)[ZT_PACKET_IDX_FLAGS] & 0xf8) | (((unsigned char)(*this)[ZT_PACKET_IDX_FLAGS] + 1) & 0x07);
 	}
 
 	/**
@@ -629,23 +635,25 @@ public:
 	 * 
 	 * @param v New packet verb
 	 */
-	inline void setVerb(Verb v) { _b[ZT_PACKET_IDX_VERB] = (char)v; }
+	inline void setVerb(Verb v) { (*this)[ZT_PACKET_IDX_VERB] = (char)v; }
 
 	/**
 	 * @return Packet verb (not including flag bits)
 	 */
-	inline Verb verb() const { return (Verb)(_b[ZT_PACKET_IDX_VERB] & 0x1f); }
+	inline Verb verb() const { return (Verb)((*this)[ZT_PACKET_IDX_VERB] & 0x1f); }
 
 	/**
 	 * @return Length of packet payload
 	 */
-	inline unsigned int payloadLength() const throw() { return ((_l < ZT_PROTO_MIN_PACKET_LENGTH) ? 0 : (_l - ZT_PROTO_MIN_PACKET_LENGTH)); }
+	inline unsigned int payloadLength() const { return ((size() < ZT_PROTO_MIN_PACKET_LENGTH) ? 0 : (size() - ZT_PROTO_MIN_PACKET_LENGTH)); }
 
 	/**
-	 * @return Packet payload
+	 * @return Raw packet payload
 	 */
-	inline unsigned char *payload() throw() { return (unsigned char *)(_b + ZT_PACKET_IDX_PAYLOAD); }
-	inline const unsigned char *payload() const throw() { return (const unsigned char *)(_b + ZT_PACKET_IDX_PAYLOAD); }
+	inline const unsigned char *payload() const
+	{
+		return field(ZT_PACKET_IDX_PAYLOAD,size() - ZT_PACKET_IDX_PAYLOAD);
+	}
 
 	/**
 	 * Compute the HMAC of this packet's payload and set HMAC field
@@ -655,13 +663,13 @@ public:
 	 * @param key 256-bit (32 byte) key
 	 */
 	inline void hmacSet(const void *key)
-		throw()
 	{
 		unsigned char mac[32];
 		unsigned char key2[32];
 		_mangleKey((const unsigned char *)key,key2);
-		HMAC::sha256(key2,sizeof(key2),_b + ZT_PACKET_IDX_VERB,(_l >= ZT_PACKET_IDX_VERB) ? (_l - ZT_PACKET_IDX_VERB) : 0,mac);
-		memcpy(_b + ZT_PACKET_IDX_HMAC,mac,8);
+		unsigned int hmacLen = (size() >= ZT_PACKET_IDX_VERB) ? (size() - ZT_PACKET_IDX_VERB) : 0;
+		HMAC::sha256(key2,sizeof(key2),field(ZT_PACKET_IDX_VERB,hmacLen),hmacLen,mac);
+		memcpy(field(ZT_PACKET_IDX_HMAC,8),mac,8);
 	}
 
 	/**
@@ -672,15 +680,15 @@ public:
 	 * @param key 256-bit (32 byte) key
 	 */
 	inline bool hmacVerify(const void *key) const
-		throw()
 	{
 		unsigned char mac[32];
 		unsigned char key2[32];
-		if (_l < ZT_PACKET_IDX_VERB)
+		if (size() < ZT_PACKET_IDX_VERB)
 			return false; // incomplete packets fail
 		_mangleKey((const unsigned char *)key,key2);
-		HMAC::sha256(key2,sizeof(key2),_b + ZT_PACKET_IDX_VERB,_l - ZT_PACKET_IDX_VERB,mac);
-		return (!memcmp(_b + ZT_PACKET_IDX_HMAC,mac,8));
+		unsigned int hmacLen = size() - ZT_PACKET_IDX_VERB;
+		HMAC::sha256(key2,sizeof(key2),field(ZT_PACKET_IDX_VERB,hmacLen),hmacLen,mac);
+		return (!memcmp(field(ZT_PACKET_IDX_HMAC,8),mac,8));
 	}
 
 	/**
@@ -689,13 +697,16 @@ public:
 	 * @param key 256-bit (32 byte) key
 	 */
 	inline void encrypt(const void *key)
-		throw()
 	{
-		_b[ZT_PACKET_IDX_FLAGS] |= ZT_PROTO_FLAG_ENCRYPTED;
+		(*this)[ZT_PACKET_IDX_FLAGS] |= ZT_PROTO_FLAG_ENCRYPTED;
 		unsigned char key2[32];
-		_mangleKey((const unsigned char *)key,key2);
-		Salsa20 s20(key2,256,_b + ZT_PACKET_IDX_IV);
-		s20.encrypt(_b + ZT_PACKET_IDX_VERB,_b + ZT_PACKET_IDX_VERB,(_l >= ZT_PACKET_IDX_VERB) ? (_l - ZT_PACKET_IDX_VERB) : 0);
+		if (size() >= ZT_PACKET_IDX_VERB) {
+			_mangleKey((const unsigned char *)key,key2);
+			Salsa20 s20(key2,256,field(ZT_PACKET_IDX_IV,8));
+			unsigned int encLen = size() - ZT_PACKET_IDX_VERB;
+			unsigned char *const encBuf = field(ZT_PACKET_IDX_VERB,encLen);
+			s20.encrypt(encBuf,encBuf,encLen);
+		}
 	}
 
 	/**
@@ -704,13 +715,16 @@ public:
 	 * @param key 256-bit (32 byte) key
 	 */
 	inline void decrypt(const void *key)
-		throw()
 	{
 		unsigned char key2[32];
-		_mangleKey((const unsigned char *)key,key2);
-		Salsa20 s20(key2,256,_b + ZT_PACKET_IDX_IV);
-		s20.decrypt(_b + ZT_PACKET_IDX_VERB,_b + ZT_PACKET_IDX_VERB,(_l >= ZT_PACKET_IDX_VERB) ? (_l - ZT_PACKET_IDX_VERB) : 0);
-		_b[ZT_PACKET_IDX_FLAGS] &= (char)(~ZT_PROTO_FLAG_ENCRYPTED);
+		if (size() >= ZT_PACKET_IDX_VERB) {
+			_mangleKey((const unsigned char *)key,key2);
+			Salsa20 s20(key2,256,field(ZT_PACKET_IDX_IV,8));
+			unsigned int decLen = size() - ZT_PACKET_IDX_VERB;
+			unsigned char *const decBuf = field(ZT_PACKET_IDX_VERB,decLen);
+			s20.decrypt(decBuf,decBuf,decLen);
+		}
+		(*this)[ZT_PACKET_IDX_FLAGS] &= (char)(~ZT_PROTO_FLAG_ENCRYPTED);
 	}
 
 	/**
@@ -724,20 +738,19 @@ public:
 	 * @return True if compression occurred
 	 */
 	inline bool compress()
-		throw()
 	{
 		unsigned char buf[ZT_PROTO_MAX_PACKET_LENGTH * 2];
-		if ((!compressed())&&(_l > (ZT_PACKET_IDX_PAYLOAD + 32))) {
-			int pl = (int)(_l - ZT_PACKET_IDX_PAYLOAD);
-			int cl = LZ4_compress((const char *)(_b + ZT_PACKET_IDX_PAYLOAD),(char *)buf,pl);
+		if ((!compressed())&&(size() > (ZT_PACKET_IDX_PAYLOAD + 32))) {
+			int pl = (int)(size() - ZT_PACKET_IDX_PAYLOAD);
+			int cl = LZ4_compress((const char *)field(ZT_PACKET_IDX_PAYLOAD,(unsigned int)pl),(char *)buf,pl);
 			if ((cl > 0)&&(cl < pl)) {
-				_b[ZT_PACKET_IDX_VERB] |= (char)ZT_PROTO_VERB_FLAG_COMPRESSED;
-				memcpy(_b + ZT_PACKET_IDX_PAYLOAD,buf,cl);
-				_l = (unsigned int)cl + ZT_PACKET_IDX_PAYLOAD;
+				(*this)[ZT_PACKET_IDX_VERB] |= (char)ZT_PROTO_VERB_FLAG_COMPRESSED;
+				setSize((unsigned int)cl + ZT_PACKET_IDX_PAYLOAD);
+				memcpy(field(ZT_PACKET_IDX_PAYLOAD,(unsigned int)cl),buf,cl);
 				return true;
 			}
 		}
-		_b[ZT_PACKET_IDX_VERB] &= (char)(~ZT_PROTO_VERB_FLAG_COMPRESSED);
+		(*this)[ZT_PACKET_IDX_VERB] &= (char)(~ZT_PROTO_VERB_FLAG_COMPRESSED);
 		return false;
 	}
 
@@ -750,18 +763,18 @@ public:
 	 * @return True if data is now decompressed and valid, false on error
 	 */
 	inline bool uncompress()
-		throw()
 	{
 		unsigned char buf[ZT_PROTO_MAX_PACKET_LENGTH];
-		if ((compressed())&&(_l >= ZT_PROTO_MIN_PACKET_LENGTH)) {
-			if (_l > ZT_PACKET_IDX_PAYLOAD) {
-				int ucl = LZ4_uncompress_unknownOutputSize((const char *)(_b + ZT_PACKET_IDX_PAYLOAD),(char *)buf,_l - ZT_PACKET_IDX_PAYLOAD,sizeof(buf));
+		if ((compressed())&&(size() >= ZT_PROTO_MIN_PACKET_LENGTH)) {
+			if (size() > ZT_PACKET_IDX_PAYLOAD) {
+				unsigned int compLen = size() - ZT_PACKET_IDX_PAYLOAD;
+				int ucl = LZ4_uncompress_unknownOutputSize((const char *)field(ZT_PACKET_IDX_PAYLOAD,compLen),(char *)buf,compLen,sizeof(buf));
 				if ((ucl > 0)&&(ucl <= (int)(capacity() - ZT_PACKET_IDX_PAYLOAD))) {
-					memcpy(_b + ZT_PACKET_IDX_PAYLOAD,buf,ucl);
-					_l = (unsigned int)ucl + ZT_PACKET_IDX_PAYLOAD;
+					setSize((unsigned int)ucl + ZT_PACKET_IDX_PAYLOAD);
+					memcpy(field(ZT_PACKET_IDX_PAYLOAD,(unsigned int)ucl),buf,ucl);
 				} else return false;
 			}
-			_b[ZT_PACKET_IDX_VERB] &= ~ZT_PROTO_VERB_FLAG_COMPRESSED;
+			(*this)[ZT_PACKET_IDX_VERB] &= ~ZT_PROTO_VERB_FLAG_COMPRESSED;
 		}
 		return true;
 	}
@@ -788,19 +801,18 @@ private:
 	 * @param out Output buffer (32 bytes)
 	 */
 	inline void _mangleKey(const unsigned char *in,unsigned char *out) const
-		throw()
 	{
 		// Random IV (Salsa20 also uses the IV natively, but HMAC doesn't), and
 		// destination and source addresses. Using dest and source addresses
 		// gives us a (likely) different key space for a->b vs b->a.
 		for(unsigned int i=0;i<18;++i) // 8 + (ZT_ADDRESS_LENGTH * 2) == 18
-			out[i] = in[i] ^ (unsigned char)_b[i];
+			out[i] = in[i] ^ (unsigned char)(*this)[i];
 		// Flags, but masking off hop count which is altered by forwarding nodes
-		out[18] = in[18] ^ ((unsigned char)_b[ZT_PACKET_IDX_FLAGS] & 0xf8);
+		out[18] = in[18] ^ ((unsigned char)(*this)[ZT_PACKET_IDX_FLAGS] & 0xf8);
 		// Raw packet size in bytes -- each raw packet size defines a possibly
 		// different space of keys.
-		out[19] = in[19] ^ (unsigned char)(_l & 0xff);
-		out[20] = in[20] ^ (unsigned char)((_l >> 8) & 0xff); // little endian
+		out[19] = in[19] ^ (unsigned char)(size() & 0xff);
+		out[20] = in[20] ^ (unsigned char)((size() >> 8) & 0xff); // little endian
 		// Rest of raw key is used unchanged
 		for(unsigned int i=21;i<32;++i)
 			out[i] = in[i];

node/Range.hpp

@@ -0,0 +1,123 @@
+/*
+ * ZeroTier One - Global Peer to Peer Ethernet
+ * Copyright (C) 2012-2013  ZeroTier Networks LLC
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * ZeroTier may be used and distributed under the terms of the GPLv3, which
+ * are available at: http://www.gnu.org/licenses/gpl-3.0.html
+ *
+ * If you would like to embed ZeroTier into a commercial application or
+ * redistribute it in a modified binary form, please contact ZeroTier Networks
+ * LLC. Start here: http://www.zerotier.com/
+ */
+
+#ifndef _ZT_RANGE_HPP
+#define _ZT_RANGE_HPP
+
+namespace ZeroTier {
+
+/**
+ * A range of numeric values
+ *
+ * @tparam T Type, can be any numeric value (int, float, double, etc.)
+ */
+template<typename T>
+class Range
+{
+public:
+	/**
+	 * Construct an empty range
+	 */
+	Range()
+		throw() :
+		start(0),
+		end(0)
+	{
+	}
+
+	/**
+	 * @param s Starting value (inclusive)
+	 * @param e Ending value (exclusive)
+	 */
+	Range(T s,T e)
+		throw() :
+		start(s),
+		end(e)
+	{
+	}
+
+	/**
+	 * Construct a range containing from n to n+1 (thus only n for integers)
+	 *
+	 * @param n Number to contain
+	 */
+	Range(T n)
+		throw() :
+		start(n),
+		end(n+1)
+	{
+	}
+
+	/**
+	 * @return end - start
+	 */
+	inline T magnitude() const
+		throw()
+	{
+		return (end - start);
+	}
+
+	/**
+	 * @return True if range contains something (magnitude is nonzero)
+	 */
+	inline operator bool() const
+		throw()
+	{
+		return (end > start);
+	}
+
+	/**
+	 * @param v Value to test
+	 * @return True if value is between start (inclusive) and end (exclusive)
+	 */
+	inline bool operator()(const T &v) const
+		throw()
+	{
+		return ((v >= start)&&(v < end));
+	}
+
+	inline bool operator==(const Range &r) const throw() { return ((start == r.start)&&(end == r.end)); }
+	inline bool operator!=(const Range &r) const throw() { return (!(*this == r)); }
+	inline bool operator<(const Range &r) const throw() { return ((start < r.start) ? true : ((start == r.start) ? (end < r.end) : false)); }
+	inline bool operator>(const Range &r) const throw() { return (r < *this); }
+	inline bool operator<=(const Range &r) const throw() { return !(r < *this); }
+	inline bool operator>=(const Range &r) const throw() { return !(*this < r); }
+
+	/**
+	 * Start of range (may be modified directly)
+	 */
+	T start;
+
+	/**
+	 * End of range (may be modified directly)
+	 */
+	T end;
+};
+
+} // namespace ZeroTier
+
+#endif

node/Salsa20.cpp

@@ -5,6 +5,7 @@
  */
 
 #include "Salsa20.hpp"
+#include "Constants.hpp"
 
 #define ROTATE(v,c) (((v) << (c)) | ((v) >> (32 - (c))))
 #define XOR(v,w) ((v) ^ (w))

node/Switch.cpp

@@ -41,6 +41,7 @@
 #include "Peer.hpp"
 #include "NodeConfig.hpp"
 #include "Demarc.hpp"
+#include "Filter.hpp"
 
 #include "../version.h"
 
@@ -49,7 +50,6 @@ namespace ZeroTier {
 Switch::Switch(const RuntimeEnvironment *renv) :
 	_r(renv)
 {
-	memset(_multicastHistory,0,sizeof(_multicastHistory));
 }
 
 Switch::~Switch()
@@ -233,7 +233,7 @@ Switch_onRemotePacket_complete_packet_handler:
 void Switch::onLocalEthernet(const SharedPtr<Network> &network,const MAC &from,const MAC &to,unsigned int etherType,const Buffer<4096> &data)
 {
 	if (from != network->tap().mac()) {
-		LOG("ignored tap: %s -> %s %s (bridging is not supported)",from.toString().c_str(),to.toString().c_str(),Utils::etherTypeName(etherType));
+		LOG("ignored tap: %s -> %s %s (bridging is not supported)",from.toString().c_str(),to.toString().c_str(),Filter::etherTypeName(etherType));
 		return;
 	}
 
@@ -245,25 +245,20 @@ void Switch::onLocalEthernet(const SharedPtr<Network> &network,const MAC &from,c
 	}
 
 	if ((etherType != ZT_ETHERTYPE_ARP)&&(etherType != ZT_ETHERTYPE_IPV4)&&(etherType != ZT_ETHERTYPE_IPV6)) {
-		LOG("ignored tap: %s -> %s %s (not a supported etherType)",from.toString().c_str(),to.toString().c_str(),Utils::etherTypeName(etherType));
+		LOG("ignored tap: %s -> %s %s (not a supported etherType)",from.toString().c_str(),to.toString().c_str(),Filter::etherTypeName(etherType));
 		return;
 	}
 
 	if (to.isMulticast()) {
 		MulticastGroup mg(to,0);
 
-		// Handle special cases: IPv4 ARP
-		if ((etherType == ZT_ETHERTYPE_ARP)&&(data.size() == 28)&&(data[2] == 0x08)&&(data[3] == 0x00)&&(data[4] == 6)&&(data[5] == 4)&&(data[7] == 0x01))
-			mg = MulticastGroup::deriveMulticastGroupForAddressResolution(InetAddress(data.field(24,4),4,0));
+		if (to.isBroadcast()) {
+			// Cram IPv4 IP into ADI field to make IPv4 ARP broadcast channel specific and scalable
+			if ((etherType == ZT_ETHERTYPE_ARP)&&(data.size() == 28)&&(data[2] == 0x08)&&(data[3] == 0x00)&&(data[4] == 6)&&(data[5] == 4)&&(data[7] == 0x01))
+				mg = MulticastGroup::deriveMulticastGroupForAddressResolution(InetAddress(data.field(24,4),4,0));
+		}
 
-		// Remember this message's CRC, but don't drop if we've already seen it
-		// since it's our own.
-		_checkAndUpdateMulticastHistory(from,mg.mac(),data.data(),data.size(),network->id(),Utils::now());
-
-		// Start multicast propagation with empty bloom filter
-		unsigned char bloom[ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE];
-		memset(bloom,0,sizeof(bloom));
-		_propagateMulticast(network,bloom,mg,0,0,from,etherType,data.data(),data.size());
+		_propagateMulticast(network,_r->identity.address(),(const unsigned char *)0,mg,0,from,etherType,data.data(),data.size());
 	} else if (to.isZeroTier()) {
 		// Simple unicast frame from us to another node
 		Address toZT(to.data + 1);
@@ -275,10 +270,10 @@ void Switch::onLocalEthernet(const SharedPtr<Network> &network,const MAC &from,c
 			outp.compress();
 			send(outp,true);
 		} else {
-			TRACE("UNICAST: %s -> %s %s (dropped, destination not a member of closed network %llu)",from.toString().c_str(),to.toString().c_str(),Utils::etherTypeName(etherType),network->id());
+			TRACE("UNICAST: %s -> %s %s (dropped, destination not a member of closed network %llu)",from.toString().c_str(),to.toString().c_str(),Filter::etherTypeName(etherType),network->id());
 		}
 	} else {
-		TRACE("UNICAST: %s -> %s %s (dropped, destination MAC not ZeroTier)",from.toString().c_str(),to.toString().c_str(),Utils::etherTypeName(etherType));
+		TRACE("UNICAST: %s -> %s %s (dropped, destination MAC not ZeroTier)",from.toString().c_str(),to.toString().c_str(),Filter::etherTypeName(etherType));
 	}
 }
 
@@ -312,6 +307,19 @@ void Switch::sendHELLO(const Address &dest)
 	send(outp,false);
 }
 
+bool Switch::sendHELLO(const SharedPtr<Peer> &dest,Demarc::Port localPort,const InetAddress &addr)
+{
+	Packet outp(dest->address(),_r->identity.address(),Packet::VERB_HELLO);
+	outp.append((unsigned char)ZT_PROTO_VERSION);
+	outp.append((unsigned char)ZEROTIER_ONE_VERSION_MAJOR);
+	outp.append((unsigned char)ZEROTIER_ONE_VERSION_MINOR);
+	outp.append((uint16_t)ZEROTIER_ONE_VERSION_REVISION);
+	outp.append(Utils::now());
+	_r->identity.serialize(outp,false);
+	outp.hmacSet(dest->macKey());
+	return _r->demarc->send(localPort,addr,outp.data(),outp.size(),-1);
+}
+
 bool Switch::unite(const Address &p1,const Address &p2,bool force)
 {
 	SharedPtr<Peer> p1p = _r->topology->getPeer(p1);
@@ -392,11 +400,8 @@ unsigned long Switch::doTimerTasks()
 			if (now >= i->second.fireAtTime) {
 				SharedPtr<Peer> withPeer = _r->topology->getPeer(i->first);
 				if (withPeer) {
-					TRACE("sending NAT-T NOP to %s(%s)",i->first.toString().c_str(),i->second.inaddr.toString().c_str());
-					Packet outp(i->first,_r->identity.address(),Packet::VERB_NOP);
-					outp.append("ZT",2); // arbitrary payload
-					outp.hmacSet(withPeer->macKey());
-					_r->demarc->send(i->second.localPort,i->second.inaddr,outp.data(),outp.size(),-1);
+					TRACE("sending NAT-T HELLO to %s(%s)",i->first.toString().c_str(),i->second.inaddr.toString().c_str());
+					sendHELLO(withPeer,i->second.localPort,i->second.inaddr);
 				}
 				_rendezvousQueue.erase(i++);
 			} else {
@@ -555,22 +560,23 @@ void Switch::_CBaddPeerFromWhois(void *arg,const SharedPtr<Peer> &p,Topology::Pe
 	}
 }
 
-void Switch::_propagateMulticast(const SharedPtr<Network> &network,unsigned char *bloom,const MulticastGroup &mg,unsigned int mcHops,unsigned int mcLoadFactor,const MAC &from,unsigned int etherType,const void *data,unsigned int len)
+void Switch::_propagateMulticast(const SharedPtr<Network> &network,const Address &upstream,const unsigned char *bloom,const MulticastGroup &mg,unsigned int mcHops,const MAC &from,unsigned int etherType,const void *data,unsigned int len)
 {
-	SharedPtr<Peer> propPeers[ZT_MULTICAST_PROPAGATION_BREADTH];
-	unsigned int np = _r->topology->pickMulticastPropagationPeers(network->id(),Address(),bloom,ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE * 8,ZT_MULTICAST_PROPAGATION_BREADTH,mg,propPeers);
+	if (mcHops > ZT_MULTICAST_PROPAGATION_DEPTH)
+		return;
 
-	for(unsigned int i=0;i<np;++i)
-		Utils::bloomAdd(bloom,ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE,propPeers[i]->address().sum());
+	Multicaster::MulticastBloomFilter newBloom(bloom); // bloom will be NULL if starting fresh
+	SharedPtr<Peer> propPeers[ZT_MULTICAST_PROPAGATION_BREADTH];
+	unsigned int np = _multicaster.pickNextPropagationPeers(*(_r->topology),network->id(),mg,upstream,newBloom,ZT_MULTICAST_PROPAGATION_BREADTH,propPeers,Utils::now());
 
 	for(unsigned int i=0;i<np;++i) {
 		Packet outp(propPeers[i]->address(),_r->identity.address(),Packet::VERB_MULTICAST_FRAME);
-		outp.append(network->id());
+		outp.append((uint64_t)network->id());
 		outp.append(mg.mac().data,6);
 		outp.append((uint32_t)mg.adi());
-		outp.append(bloom,ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE);
+		outp.append(newBloom.data(),ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE);
 		outp.append((uint8_t)mcHops);
-		outp.append((uint16_t)mcLoadFactor);
+		outp.append((unsigned char)0,2); // reserved, 0
 		outp.append(from.data,6);
 		outp.append((uint16_t)etherType);
 		outp.append(data,len);
@@ -748,27 +754,22 @@ Switch::PacketServiceAttemptResult Switch::_tryHandleRemotePacket(Demarc::Port l
 						if (network->isAllowed(source)) {
 							if (packet.size() > ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD) {
 								MulticastGroup mg(MAC(packet.field(ZT_PROTO_VERB_MULTICAST_FRAME_IDX_MULTICAST_MAC,6)),packet.at<uint32_t>(ZT_PROTO_VERB_MULTICAST_FRAME_IDX_ADI));
-								unsigned char bloom[ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE];
-								memcpy(bloom,packet.field(ZT_PROTO_VERB_MULTICAST_FRAME_IDX_BLOOM,ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE),ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE);
 								unsigned int hops = packet[ZT_PROTO_VERB_MULTICAST_FRAME_IDX_HOPS];
-								unsigned int loadFactor = packet.at<uint16_t>(ZT_PROTO_VERB_MULTICAST_FRAME_IDX_LOAD_FACTOR);
 								MAC fromMac(packet.field(ZT_PROTO_VERB_MULTICAST_FRAME_IDX_FROM_MAC,6));
 								unsigned int etherType = packet.at<uint16_t>(ZT_PROTO_VERB_MULTICAST_FRAME_IDX_ETHERTYPE);
+								unsigned int payloadLen = packet.size() - ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD;
+								unsigned char *payload = packet.field(ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD,payloadLen);
 
-								if ((fromMac.isZeroTier())&&(network->isAllowed(Address(fromMac)))) {
-									if (_checkAndUpdateMulticastHistory(fromMac,mg.mac(),packet.data() + ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD,packet.size() - ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD,network->id(),now)) {
-										TRACE("dropped MULTICAST_FRAME from %s: duplicate multicast",source.toString().c_str());
+								if (fromMac == network->tap().mac()) {
+									TRACE("dropped boomerang MULTICAST_FRAME from %s",source.toString().c_str());
+								} if (network->isAllowed(fromMac)) {
+									if (_multicaster.checkAndUpdateMulticastHistory(fromMac,mg,payload,payloadLen,network->id(),now)) {
+										// TODO: check if allowed etherType
+										network->tap().put(fromMac,mg.mac(),etherType,payload,payloadLen);
 									} else {
-										//TRACE("MULTICAST_FRAME: %s -> %s (adi: %.8lx), %u bytes, net: %llu",fromMac.toString().c_str(),mg.mac().toString().c_str(),(unsigned long)mg.adi(),packet.size() - ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD,network->id());
-										network->tap().put(fromMac,mg.mac(),etherType,packet.data() + ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD,packet.size() - ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD);
-
-										// TODO: implement load factor based propagation rate limitation
-										// How it will work: each node will adjust loadFactor based on
-										// its current load of multicast traffic. Then it will probabilistically
-										// fail to propagate, with the probability being based on load factor.
-										// This will need some in-the-field testing and tuning to get right.
-										_propagateMulticast(network,bloom,mg,hops+1,loadFactor,fromMac,etherType,packet.data() + ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD,packet.size() - ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD);
+										TRACE("duplicate MULTICAST_FRAME from %s: %s -> %s (adi: %.8lx), %u bytes, net: %llu",source.toString().c_str(),fromMac.toString().c_str(),mg.mac().toString().c_str(),(unsigned long)mg.adi(),packet.size() - ZT_PROTO_VERB_MULTICAST_FRAME_IDX_PAYLOAD,network->id());
 									}
+									_propagateMulticast(network,source,packet.field(ZT_PROTO_VERB_MULTICAST_FRAME_IDX_BLOOM,ZT_PROTO_VERB_MULTICAST_FRAME_BLOOM_FILTER_SIZE),mg,hops+1,fromMac,etherType,payload,payloadLen);
 								} else {
 									TRACE("dropped MULTICAST_FRAME from %s: ultimate sender %s not a member of closed network %llu",source.toString().c_str(),fromMac.toString().c_str(),network->id());
 								}
@@ -797,7 +798,7 @@ Switch::PacketServiceAttemptResult Switch::_tryHandleRemotePacket(Demarc::Port l
 								MAC mac(packet.field(ptr,6)); ptr += 6;
 								uint32_t adi = packet.at<uint32_t>(ptr); ptr += 4;
 								TRACE("peer %s likes multicast group %s:%.8lx on network %llu",source.toString().c_str(),mac.toString().c_str(),(unsigned long)adi,nwid);
-								_r->topology->likesMulticastGroup(nwid,MulticastGroup(mac,adi),source,now);
+								_multicaster.likesMulticastGroup(nwid,MulticastGroup(mac,adi),source,now);
 								++numAccepted;
 							} else {
 								TRACE("ignored MULTICAST_LIKE from %s: not a member of closed network %llu",source.toString().c_str(),nwid);

node/Switch.hpp

@@ -44,6 +44,7 @@
 #include "Network.hpp"
 #include "SharedPtr.hpp"
 #include "Demarc.hpp"
+#include "Multicaster.hpp"
 
 namespace ZeroTier {
 
@@ -106,6 +107,16 @@ public:
 	 */
 	void sendHELLO(const Address &dest);
 
+	/**
+	 * Send a HELLO announcement immediately to the indicated address
+	 *
+	 * @param localPort Originating local port or ANY_PORT to pick
+	 * @param addr IP address to send to
+	 * @param dest Destination peer
+	 * @return True if send appears successful
+	 */
+	bool sendHELLO(const SharedPtr<Peer> &dest,Demarc::Port localPort,const InetAddress &addr);
+
 	/**
 	 * Send RENDEZVOUS to two peers to permit them to directly connect
 	 *
@@ -161,7 +172,7 @@ private:
 	static void _CBaddPeerFromHello(void *arg,const SharedPtr<Peer> &p,Topology::PeerVerifyResult result);
 	static void _CBaddPeerFromWhois(void *arg,const SharedPtr<Peer> &p,Topology::PeerVerifyResult result); // arg == this
 
-	void _propagateMulticast(const SharedPtr<Network> &network,unsigned char *bloom,const MulticastGroup &mg,unsigned int mcHops,unsigned int mcLoadFactor,const MAC &from,unsigned int etherType,const void *data,unsigned int len);
+	void _propagateMulticast(const SharedPtr<Network> &network,const Address &upstream,const unsigned char *bloom,const MulticastGroup &mg,unsigned int mcHops,const MAC &from,unsigned int etherType,const void *data,unsigned int len);
 	PacketServiceAttemptResult _tryHandleRemotePacket(Demarc::Port localPort,const InetAddress &fromAddr,Packet &packet);
 	void _doHELLO(Demarc::Port localPort,const InetAddress &fromAddr,Packet &packet);
 	void _requestWhois(const Address &addr);
@@ -169,39 +180,9 @@ private:
 	PacketServiceAttemptResult _trySend(const Packet &packet,bool encrypt);
 	void _retryPendingFor(const Address &addr);
 
-	// Updates entry for crc in multicast history, returns true if already
-	// present in history and not expired.
-	inline bool _checkAndUpdateMulticastHistory(const MAC &fromMac,const MAC &toMulticastMac,const void *payload,unsigned int len,const uint64_t nwid,const uint64_t now)
-	{
-		uint64_t crc = Utils::crc64(0,fromMac.data,6);
-		crc = Utils::crc64(crc,toMulticastMac.data,6);
-		crc = Utils::crc64(crc,payload,len);
-		crc += nwid; // also include network ID
-
-		uint64_t earliest = 0xffffffffffffffffULL;
-		unsigned long earliestIdx = 0;
-		for(unsigned int i=0;i<ZT_MULTICAST_DEDUP_HISTORY_LENGTH;++i) {
-			if (_multicastHistory[i][0] == crc) {
-				uint64_t then = _multicastHistory[i][1];
-				_multicastHistory[i][1] = now;
-				return ((now - then) < ZT_MULTICAST_DEDUP_HISTORY_EXPIRE);
-			} else if (_multicastHistory[i][1] < earliest) {
-				earliest = _multicastHistory[i][1];
-				earliestIdx = i;
-			}
-		}
-
-		_multicastHistory[earliestIdx][0] = crc; // replace oldest entry
-		_multicastHistory[earliestIdx][1] = now;
-
-		return false;
-	}
-
 	const RuntimeEnvironment *const _r;
 
-	// Multicast packet CRC64's for packets we've received recently, to reject
-	// duplicates during propagation. [0] is CRC64, [1] is time.
-	uint64_t _multicastHistory[ZT_MULTICAST_DEDUP_HISTORY_LENGTH][2];
+	Multicaster _multicaster;
 
 	struct WhoisRequest
 	{