Maybe I have a fundamental misunderstanding of how Tailscale works, but I always feel like there is a disconnect in how it is received on HN.
Usually people are pretty critical/cynical of sending sensitive data to a closed source third party server, no matter how strong their claims of 'being the good guys' are (eg see Telegram).
But somehow we're all meant to be happy giving full control of our entire network to a commercial company running a closed source command and control server?
And even if everything is secure right now, you cannot guarantee this stays the case in the future. By using the "tailscale ecosystem" you are locking yourself into a provider that can change the ecosystem (clients, services, servers) or put things behind a paywalls anytime. Or create add-ons that are useful and no longer privacy-preserving. The fact that they are a VC-funded business makes me believe this is how the company will end up: Customer data will be monetized in one way or another. How else are going to create returns for shareholders that justify their valuation? Certainly not by open sourcing stuff and not looking at your data. We've seen these VC incentives play out again and again in other companies.
Not really true, tailscale clients do allow you to point to different control servers and open source implementations do exist[1] and are thriving. The clients are also open source and you can even create one yourself if you are willing to.
The thing is, what makes tailscale works really well as a "central" control server is that it makes a lot easier to connect your personal machines. You don't need to deploy your own server, or mess with networking stuff. You just download it, log-in and there you go. I myself have invited some non-tech friends to my network for playing lan games from time time and they find pretty easy to setup tailscale on their side.
Once a FOSS ecosystem exists, it can always be maintained as such.
What may theoretically happen is that the same client won't be able to join both an open-source controller and a controller from Tailscale-turned-evil.
I suppose that corporations, especially those without huge IT departments, will and do happily pay Tailscale the reasonable money, and have their secure VPN just working. For them, it's no different than paying for Zoom or Office365, only cheaper. They totally do not want to depend on in-house networking expertise.
So I think Tailscale will be doing well financially,
They could, but let's cross that bridge when we get to it. You're trying to justify the dislike of a product because of what they might do. That's a pretty weak argument.
For us. For the folks who browse HN all day, yeah. But have you tried getting a non-technologist to use it? I set a friend up with Tailscale between her Synology and laptop and it was a breeze - something I could do over the phone. For me getting Wireguard set up wasn't tricky, but I definitely leaned on Google some, and I would argue I know what I'm doing.
I would love for a company to release bridge as open source that I could deploy on a VM somewhere but still have it be Tailscale easy for normal folks, but there's no money in that model.
I fail to see how that is any different to other modern network control solutions out there? Every major network vendor is moving towards a cloud control service of some kind.
Further, Honestly at this point I would trust TailScale over say OpenVPN, or Cisco, or .....
Road 1: Sale to usual suspects like Palo Alto (though that window is closing due to raising $100M) or Cisco (that window may close if they raise again?). It is basically modern vpn, though will be years with a big enterprise culture reset & a consumer tier for that to become true. They will run out of acquirers soon who would have the incentive to overpay, eg, if they raise more and narrows down to say oracle, ms, apple, and google, not sure why any would buy vs build. Hopefully they will not dip much into the funds so they can cleanly exit without forcing an acquirer to screw over their users. (See: Evernote)
Road 2: IPO and become a platform for bigger stuff... Like end-user-friendly VPN. Who knows, but good luck! Flipping to 'real' enterprise sales and figuring out the consumer tiers are big culture shifts, but luckily... Hireable.
Meanwhile, growing just with more niche/skilled Linux power user teams gets them far -- the compliance checkbox is huge for growth, see Drata and Vanta -- so am not worried :)
Agreed with the OSS concern so we decided against putting them in the critical path of our enterprise offering (a shame!), but as an internal tool, it looks great!
In many situations it's not just hard, it's outright impossible.
For example, how would you connect two Raspberry Pis between two CG-NATted internet connections using Wireguard, without resorting to setting up a publicly reachable VPN server?
If you have a public and at least semi-stable IPv4 address and control your firewall/NAT, great. But unfortunately less and less people do, these days.
Yes, if you have it everywhere you want to host services and everywhere you want to access these services, that's great. Realistically, I think it's going to be decades until an IPv6-only service is feasible.
> Otherwise, NAT traversal techniques such as STUN, or hole-punching.
Neither of which Wireguard supports out of the box. To be clear, it absolutely shouldn't – it's a different concern, and the appeal of Wireguard is specifically that it isn't trying to do everything and the kitchen sink.
So, is there an easy-to-use NAT traversal orchestration service for Wireguard out there that isn't Tailscale?
I love WG and use it extensively, but the attraction of Tailscale (and why I use it) is that it takes disparate concepts and gives you a nice visual control panel to manage them and see the status of all your devices in one place.
Nothing. Absolutely nothing exists for Wireguard that does even some of that, which doesn't also cost money.
I don't think Wireguard is even capable of the 'DERP' server concept to get around NAT limitations.
So no, you can't 'just use Wireguard' to accomplish what Tailscale does.
Spoken like somebody who has never had a "Can you help me get my microsoft wifi [Spectrum cable internet] to make sure my gmail [person@icloud.com] doesn't send my retirement [tax refund] to the hackers" type call.
>And even if everything is secure right now, you cannot guarantee this stays the case in the future. By using the "tailscale ecosystem" you are locking yourself into a provider that can change the ecosystem (clients, services, servers) or put things behind a paywalls anytime.
I can't (and refuse) to live my lie by "what ifs." If they do any of those things, it's easy enough to pivot to the 3 or 4 other providers our there, like ZeroTier, who offer the same thing.
My hope is if they do, the FOSS community will have gotten their act together and built a capable replacement, which is usually the case.
At this point you could stir up hysteria over anything. Your cloud provider, ISP, operating system support team, XYZ SaaS provider could also just invent a new billing policy to screw the customer over.
At the same time, ^ the big providers have profitable revenue streams, so they don't have much incentive to change their billing. VC startups of course on growing instead of verifying that their business model actually works, and end up either:
- locking the software behind a paywall
- locking the software behind a paywall
- inventing a proprietary + open-source + pay us royalties license
- pretending that their software is free whilst employing a proprietary + pay us royalties for anything bigger than a hobby project license
I remember reading that Tailscale is "helping out" [1] development of Headscale [2], an open-source re-implementation of their command server so that the two remain compatible as new features are added to the official one.
Leaving aside adoption, there is a degree to which HN is impressed that Tailscale has taken a set of technical problems that have caused many of us pain over the years and that drive some less than ideal setups and just made them go away. They are magically taken care of behind an easy to use setup that Just Works. There's basically none of the "you have to tweak this setting under this circumstance" needed to get it working. That is difficult engineering requiring people who understand the problem domain and have a clear picture of the right architecture, as well as good product engineering.
Yes. It is just Wireguard… with a bit of magic on top.
As far as I can tell, the Tailscale magic is maintaining and distributing a wg.conf that is setup for a dynamic mesh Wireguard network. To do this (and in particular the NAT traversal aspect), they use their own software to setup and figure out NAT ports.
One you have Wireguard figured out, it’s pretty easy to setup a hub and spoke model VPN, especially if you have a static server somewhere. Setting up a dynamic mesh VPN is another thing entirely and doing it without requiring a server from Tailscale to be part of the network is, like I said, a bit of magic. It’s a straightforward setup that they explain quite well on their site. But to actually make it work is quite impressive to me.
AFAICT, Tailscale does not route your traffic through its servers (because it would be costly, among other things), but it does control your VPN nodes to distribute all the configuration fairy dust. So a server from Tailscale is still needed (and billed for).
Right, the command and control server -- but that server does not need to be part of your private Tailscale/WG network. The Tailscale server is only required for coordinating the WG configuration. My point was that this was all done without the Tailscale server needing to actually participate in your private network. That is to say, the Tailscale server can't monitor your traffic or private servers.
Except of course in the case of Funnel... which is the original subject here. In this case, the Tailscale ingress server is added to your private network/nodelist. This it so that it can communicate with whatever private service you are running and then proxy that back out to the public Internet.
I think tailscale may be the only company that is build on WireGuard but also commits code to wireguard repo itself.
That says a lot, to me at least, but parent's comment on giving the keys to the kingdom to a third party still holds (as it does for many other SaaS/cloud/hosted platforms of course).
Um. I already have like 100s of WG connection. Office to cloud, various servers all over connected to others, servers on many WG "subnets".
WG is the easiest VPN I've ever used. Dropped all these other crazy rigs (stunnel, ssh-tunnel, etc) cause WG always "just works" and is on every platform.
However, I've spent loads of time as network admin, planning IP-space for VPN layers, first time in 1997.
I agree it's not ideal, but I can tell you why I'm excited about things like[0] Tailscale, Cloudflare Tunnel, ngrok, etc.
They enable you to move your selfhosted services from expensive, slow VPSes you don't control to fast devices in your own home[4]. IMO this is strictly better than a VPS in terms of privacy and data control. It's a step in the right direction, back towards the initial intent of the internet, but also forward with the lessons we've learned in the real world.
The reality today is that selfhosting is way too hard[1]. It shouldn't be any more complicated or less secure than running an app on your phone.
I think services like Tailscale are going to enable the first generation of selfhosting that approaches that level of simplicity. Once the market is proven, the second generation is going be designed for selfhosters and have features like end-to-end encryption, domain name integration, and simple GUI interfaces.
The other key pieces are strong sandboxing, which is now possible on all major desktop OSes through virtualization (mobile is coming[2]), and dead-simple cloud backups.
The technology for all these things exists, it just hasn't been integrated yet.
> They enable you to move your selfhosted services from expensive, slow VPSes you don't control to fast devices in your own home[4]. IMO this is strictly better than a VPS in terms of privacy and data control.
How is trusting your network entry point to Cloudflare (or whoever) any better than having it at a VPS? At least with the VPS you know what's happening inside the box.
Here's a better solution: Get a free/cheap VPS. Setup a Wireguard tunnel from your home server to it. Slap a reverse proxy on the VPS that forwards internet traffic through the tunnel to your home server.
> How is trusting your network entry point to Cloudflare (or whoever) any better than having it at a VPS?
I was referring to the other advantages, not the privacy. They're about the same on privacy.
> Here's a better solution: Get a free/cheap VPS. Setup a Wireguard tunnel from your home server to it. Slap a reverse proxy on the VPS that forwards internet traffic through the tunnel to your home server.
Way too difficult for the users I'm trying to reach.
Strong sandboxing, yes. But I'd especially appreciate it if it was wrapped up in an interface like Sandstorm.io.
I want to teach something like Tailscale about my contacts, so they are allowed to communicate with my services (through their services).
Then I want to run an app inside a strong sanbox that has capability-based APIs to do things kind of like ZeroMQ for sending messages TO THE CONTACTS. I don't want the API to look like a tcp/ip or udp connection. I want the app to think it's sending a message to a contact I've given it permission to talk to.
If this was running in a strong sandbox, I think this would be an awesome way to develop and use simple federated apps. If something like Mastodon existed on top of this, I'd think it would be really secure and much easier to tell people to stand up their own node...
I consider Sandstorm to sort of be the platonic ideal in many ways. But in practice, the fact that the apps have to be modified to run on it holds it back.
Sure, but I feel like that's just part of the trade-offs. And chicken-and-egg, really.
But if I were really bought in to Sandstorm.io, developing new apps for it would make a lot of sense, and I agree with their assessments about accounts and security - that those are better left to the OS.
So I'm just wanting to build federated apps, using that same philosophy.
And if Tailscale Funnel let me safely run in a sandbox on prem...?
That sounds amazing.
Like, this is the kind of thing where I'd buy a headless PC just to host federated apps for my extended friends and family network, if the config were painless enough and the sandbox secure enough, and the self-updating, self-maintaining story of Sandstorm.io worked well enough...
It's like the Chrome OS of federation, in my head. The OS does exactly and only what you need, making it easy for you to add the business logic of your thing.
Like, I wish this was roughly how SAS worked...
I wish to hell more businesses felt secure in letting their employees stand up convenience servers. And I feel like these are steps in the right direction.
Honestly, I hate the idea of having a middle man, but having tried and researched extensively how to make something like a direct tunnel between two clients over the internet it just doesn't always work.
NAT is a godsend for IPv4 exhaustion, but it's also fundamentally crippled the ability for people to host things or make things available directly from their homes.
Hole-punching is an inexact process due to the variety of different NAT types, some of which (e.g. Carrier-grade) simply do not allow that sort of connection. So there must be a middle man that accepts packets on their publicly available port and passes it on to another established connection. TURN/STUN (et. al.) exist but are archaic and do the same thing but with less accountability.
I hate it too but until we have IPv6 by default with user controlled firewalls hosting something in your garage without a business line is not feasible. Hell I have a 5$ a month VPS purely so it can act as the middle man to the servers in my home. At least then I only need to trust myself as the middle man.
Their middle man in the data plane handles encrypted packets so that's not the problem here.
The problem is their control plane that controls the encryption keys. A malicious admin inside TS (or a hack) could grant itself membership in any of their customer's networks. (Or at least this is the worry I read from GP)
That's definitely a concern, but I feel this can be mitigated by running your own network on top of theirs. Anyone in my home is part of my network, doesn't mean they're in the wg network too.
Aside from that, it's definitely a problem that they could include themselves in any customer network, but the accountability still stands. If someone got in without your screw-up, at least you know who to point the finger at once the dust settles.
I'd argue it should be treated as a base to overlay your network on top of. Although admittedly I say that as someone that doesn't use their services for similar reasons.
> If someone got in without your screw-up, at least you know who to point the finger at once the dust settles.
How do you know you didn't screw up? There are so many vulnerabilities in the gazillion or random stuff you run every day on your laptop. I'd argue it's more likely that something like that was breached than Tailscaled was breached or rogue.
That I don't do, because the coordination server, the relay system (which you can also self-host), and the server side UI are really good.
And also the public behaviour of the persons working at Tailscale as well as Tailscale's approach towards FOSS generally increase my level of trust in them. IOW they strike me as Nice Folks(TM), and if Nice Folks(TM) don't inspire confidence to you then you probably want to run the whole thing as described above.
I mean, please read this in its entirety. They even have a "Encouraging Headscale" section.
> I mean, please read this in its entirety. They even have a "Encouraging Headscale" section.
Honestly that makes it even weirder.
Usually companies will just not acknowledge that an open alternative that plugs into their existing product exists, and if they do it's for enforcement/diverting purposes ("Don't use this, please stick to the official stuff")
If you're at a point where you acknowledge, accept, and even help the unofficial FOSS alternative, why not make your official stuff the same way?
Product differentiation. If you run the head scale server, I’m not certain the funnel service will work. And there’s probably other stuff that is not planned to work with the FOSS coordinator server. And of course, one of the big things is: they don’t have to support anything with headscale, even if it’s sort of their software. Use the official stuff, and you get support. Pay, and you get better support.
For that matter one can use Windows’s built-in WSL or Hyper-V functionality to run the Linux tailscale client, too.
In fact, I do this anyway because their Windows client’s subnet router functionality is documented as being less performant than their FOSS client. I use my home desktop running Windows as a subnet router from tailscale into my home network, but when I read the note about performance I quickly installed Alpine Linux on a Hyper-V virtual machine and installed tailscale on there.
That simple Alpine VM takes up so little resources that I don’t even notice any impact on performance with my 2009 12 core Xeon system. (This was a huge system, but I would imagine a modern low-midrange CPU would easily outperform it and likewise handle a micro-VM just as well)
One static route added to my home router, and traffic flows both directions just fine. No need to install the client on any more machines on my network. This also saves on licensing, helping me stay within my free plan. :)
Maybe it doesn't make sense to you, and you may not agree with it, but if they state that this is the reason why they do it this way, then we should at least consider that it might actually be the reason why they do it this way.
I see no reason why the tidying couldn't be done in public. Why not make a public repo that's closed off to external influence until it's "tidy" enough?
If they use a proprietary third party library, they might not have a source distribution license of said library. The same way that you can't just unilaterally decalare an MIT licensed project is GPL without ensuring that all of the code was dual licensed or covered under a CLA.
EDIT: I saw another comment that says that it's "just" the GUI applications for Windows and Mac that aren't open, but the core functionality is. I know that in Windows-land, it's _very_ common to use proprietary UI libraries like Telerik UI[0], or devexpress [1]
I've had the same thought re: iOS/macOS clients. Oh how I would love to dive into those small codebases and add some simple QOL stuff. I've been watching but they don't seem to be adding any Apple-platforms focused roles/people, which is fine, but I wanna work at Tailscale…
I think that by the sheer nature of Wireguard, it doesn't matter much. We don't send any readable data to Tailscale, they, for the greater part, handle plumbing between nodes. What goes in the pipes remains unnoticed and unknown to them.
Their MagicDNS feature may raise different concerns though, but I'll let others comment on it.
This new service allows them (or if they’re hacked - anyone)… to MitM your connections - they state themselves they could ssl terminate the connections as they own the ts.net level, they say they don’t but that’s now…
> This new service allows them (or if they’re hacked - anyone)… to MitM your connections...
TFA mentions that the immutable certificate logs reveal if something spooky is going on. Not enough of a guarantee since the tailscale client may siphon off certs from a local device to its servers anyway. But that's us being extremely paranoid about tailscale (in which case, why use it?).
Second, most services are run behind reverse-proxying load balancers which invariably are setup to "MiTM" aka terminate TLS. The providers running those load balancers control IPs + DNS records too, for good measure.
I guess, supporting a dizzying array of functionality makes Tailscale a decentralised cloud service provider themselves.
Why depend on Tailscale when you can go 100% open source and use slack's nebula or plain old wireguard or one of those open source wireguard manager apps.
Same here. Also, in my mind, wireguard just works. I tried setting up OpenVPN one time and gave up. Wireguard was a breath of fresh air. Generated some QR codes for my mobile devices, rsynced some .conf files to my machines and I was good to go.
This is true, on my iPhone 12 mini Tailscale is often the heaviest app (now at 17%) whereas Wireguard never really registered. Strange thing is that on some days Tailscale also barely registers. Could be that the Home Assistant app, which also takes a lot off battery connects via the Tailnet, and it communicates a lot in the background (tracks my position and some other Phone sensors.)
Nebula seems to be under Defined Networking (https://www.defined.net/) now. It lacks the ease and tooling of the multitude of others that emerged in the space. My gut is that it's usable for the clever home user but meant for a professional IT dept with the ability/desire to build their own tooling and automation around it. The spartan mobile app is a sign of such. Tailscale is probably more easily usable for the average user than nebula or even vanilla wireguard.
Former Wireguard user here. It's fantastic for point to point VPN but I moved to Tailscale last week and it's so much better having all of my devices on one flat network. Not to mention getting my parents devices on there with no config.
>But somehow we're all meant to be happy giving full control of our entire network to a commercial company running a closed source command and control server?
Yes, Tailscale is THAT good this tradeoff is worth it.
This is huge. One of the last major missing features from Tailscale IMO. I maintain a list of tunneling solutions[0]. Personally, I think the future of p2p networking and selfhosting may be through tunneled, SNI-routed TLS connections (exactly what Tailscale just announced). It solves IP exhaustion, NAT, and IP privacy at the cost of an extra hop and no UDP.
The big question is going to be pricing. The current top player in this space is Cloudflare Tunnel, which is a loss-leader product that technically forbids selfhosting anything other than HTML sites.
Selfhosting media can use tons of bandwidth. Any service that doesn't charge per GB is incentivized to limit your speeds.
> It solves IP exhaustion, NAT, and IP privacy at the cost of an extra hop and no UDP.
That is awfully expensive for something ipv6 already solves minus the privacy part. I don't see how it can be considered "huge". A slight convenience maybe?
Also, routing everything through a 3rd party is a massive downside.
IPv6 the technology solves it, IPv6 as it exists in the real world does not. Even if we got 100% IPv6 adoption overnight, you would still not have a universally adopted API for punching holes through router firewalls for p2p applications.
Yep, and such a thing would never be allowed anyway because what network administrator will look at “yes random client devices can punch holes in my network from the outside to your not very securely assigned address” and rightfully say nope.
Here’s the attack:
- You want to get to A
- And you’re on a device B that has access to hole-punch-as-a-service but no direct access to A. Good network administrator / sysadmin!
- Attack NDP (ie ARP spoofing but IPv6 flavored) to trick the firewall just for a moment that you’re A [1] and request the port be opened to your evil server C on the public internet.
Yep, tunneling is way more secure than direct p2p. Especially if you run the service inside some sort of sandbox/container/VM which is possible on all major OSes these days.
This can have quite an effect on sync speed. I set it up for my dad and initially his PC and laptop were syncing incredibly slowly. It turned out they both thought his home network was a public one, so everything was going out to a relay server and back.
UPnP needs management. I'd like to be able to set rules about what ports can be opened by UPnP, from where, by whom, with or without a notification in email/logs/alerts etc.
So basically that would mean a cloudflare for P2P, though maintaining data privacy at least.
It’s better than no P2P but IPv6 solves exhaustion and NAT without the performance hit or protocol limitations and with no extra third party intermediary in the way.
BTW this maintains data privacy but you can still tell a whole whole lot from metadata.
On the flip side it would prevent the kind of “griefing” with DDOS that happens every once in a while with self hosted and P2P things. It’s not that common unless you are engaging with certain communities but it is an inherent Internet architectural flaw that this kind of works around (at a cost).
I envision something more decentralized than Cloudflare[0]. I think you could have local companies in every major city that would own a block of IP addresses and provide tunneling services with SNI routing for people in the same region. They can also provide more expensive services like DDoS protection and caching for popular sites or even on-demand when your mom blog goes viral.
> It’s better than no P2P but IPv6 solves exhaustion and NAT without the performance hit or protocol limitations and with no extra third party intermediary in the way.
I just don't see ISPs ever implementing IPv6 without governments forcing them to. It enables p2p which increases upload bandwidth requirements and cuts into their bottom line. And even if we get IPv6 you still need the ability to open firewall ports in a way the average user can understand. Not hard technically but that's another standard that everyone is going to have to agree on and implement.
[0]: Which I know you can appreciate. "Decentralize until it hurts; centralize until it works" is still one of my mottoes.
Doesn't need to be a conspiracy, just misaligned incentives. That said, I do agree that customers aren't clamoring for upload bandwidth and p2p, because the software for selfhosting is still way too complicated and insecure for the average user to set up and run. It's a chicken/egg problem, and IMO the software needs to get better first (which is what I'm working on).
All of the home ISPs I've used in the past 10 years have IPv6. I know a few don't, including some surprising ones such as new FTTH, but they all say it's "coming."
Once you wire fiber upstream bandwidth doesn't matter that much. ISPs are directly peered into the core of the Internet and pay bulk peering bandwidth pricing not cloud bandwidth pricing. Cloud bandwidth pricing is completely insane, like 10000% markup or more in many cases, and the fact that they charge more for outbound is just a strategy to make data flow into their cloud but not out. Bandwidth is actually dirt cheap or companies like Cloudflare could never exist.
The main reason they drag their feet on IPv6 is that there's no strong forcing function and dual stack adds complexity. When they're on a deadline to ship they punt on it. They are starting to get complaints from gamers though. Gamers tend to be the top home users that drive early adoption of all kinds of things.
Edit: another funny thing is that most ISP/telecom stuff these days is IPv6 at the core. If they're not doing it at the edge for new stuff it means they've just got it off and are running IPv4 over IPv6 within their network most of the time. This is even more true for mobile networks, some of which are IPv6 only and the phone does 4to6.
The rollout is still happening... slloooooooowly... The Internet is huge and it takes forever to do a major upgrade like this. I wish they'd have just made IP 48 or 64 bit with an easier upgrade path, but that's water under the bridge now.
This is hopeful. I agree there's a good chance we'll all be running on IPv6 eventually. I just don't see it happening in the next 10 years unless the gamers can get us there. Personally, I'm hoping to see a selfhosting revolution in the next 5 years, and I think that's going to happen over tunnels and overlay networks.
It'll happen over overlay networks for other reasons, like the Internet being a dark forest hellscape where anything public gets attacked.
Overlay networks are probably the future. IPv6 lets them be fully P2P and more decentralized.
Edit: one more piece of detail about upstream bandwidth at home. 30-40mbps/sec up is a DOCSIS cable system limit. That's the only reason for that really. Cable was never designed to be bidirectional and the stuff they do with modulation to get shared upstream is heroic. It's also related to why cablemodems tend to use a lot of power. They have to blast spread spectrum pretty loud to get it to the CO. Fiber will save energy.
I just tried turning off wifi on my phone. After googling what is my ip and pinging the IPv6 address from another IPv6 device, it doesn't work. Whether it's a firewall/NAT/whatever is irrelevant. IPv6 as deployed does not provide p2p.
PS - typing the IPv6 address was super annoying and error prone compared to IPv4.
Your mobile network is probably still V6 at the core. They're either giving you V4 at the endpoint or your phone is hiding V6 from you. Android or iOS? Apple stuff is V6 native now. Android's gonna be mixed.
Depends on the carrier network though. I am on Spectrum (Verizon network) in Ohio and get V6 with CGN for V4. My wired ISP provides dual stack.
Never tried but I somehow doubt it. Standard issue UDP hole punching absolutely does work though.
You still usually need hole punching with IPv6 if there's any kind of firewall in the way, but unlike IPv4 with NAT it's virtually 100% successful. Even if IPv6 NAT is deployed there's usually a 1:1 internal/external IP mapping making hole punching 100% successful.
Phone data traffic is proxied or NATted for qos and other purposes, even with IPv6.
It works better outbound than the old model though. I use iSH to get a Linux shell and with a mini Bluetooth keyboard I work on some hobby projects on the train from my iPhone.
Agreed, this is awesome! I've been using Cloudflare Tunnel for local dev, but the fact that it seems to be coupled to their CDN product with no way to permanently turn off the CDN functionality has caused quite a few headaches lately (though it's possible to turn off temporarily using "Development Mode").
Would love to give this a try, though from the post it's not clear if we could use our own custom domains instead of the provided ts.net ones? This is a necessity for my use case where I need to be able to handle wildcard subdomains.
Where do you see Cloudflare Tunnel T&C? Their help documentation seems to promote Tunnel/Access apps in every possible scenario:
> Our connector, cloudflared, was designed to be lightweight and flexible enough to be effectively deployed on Raspberry Pi, your laptop or a server running your data center. Tunnel does not programmatically enforce any throughput limitations.
> If you are hosting a Tunnel in GCP, AWS, or Azure you can view our deployment guides which are more prescriptive in assigning minimum system requirements.
Tailscale and the people building it continue to blow me away. The ingenuity, reliability, polish, and speed of feature delivery are unparalleled in software.
I'm eager to get rid of needing DDNS and open ports just for web hooks. And I can "flatten" my stack by cutting out a reverse proxy / weird port forwarding stuff.
Sure. I don't think tailscale does anything that's impossible through other means (cue comments about how anyone can recreate Dropbox with an FTP server, svn, and a fuse file system). IMO whats great about it is that it's easy to set up and is incredibly low maintenance (at least they way I use it).
Further, and to their credit theres a lot of great tech in there.
Just the other day I stumbled on DERP [1] their wireguard-over-tcp (over their node network) implementation which they fall back to if NAT traversal fails. While sure, this is partly ipv4 problem, it recently helped me connect with friends behind government firewalls, presumably because the packets aren't recognized yet.
But that sort of automated connectivity during nat traversal failure through a global network of edges isn't something you can easily setup yourself (if you care).
The 7% is how often it ends up helping after all the traditional IPv4 traversal tricks.
We race all the options and see which works & is fastest. It's likely it's more like 16% or more (I don't have that number handy) in practice but 7% is just how often it would've not worked at all and had to go to DERP relays if it weren't for IPv6 being an option.
I've been using tailscale in my home setup for a while and really appreciate the simplicity. It has just worked in my experience. Before taking the leap for tailscale I was semi-struggling with a vault + wireguard + consul-template approach which was pretty cool and fun to setup but a bit unnerving to be unsure if I got anything wrong and the chances of me being exposed.
This looks like yet another feature that fits my use case and reduces my security burden. Though learning a few things from this post the first being that I should have replaced my use of haproxy with rinetd ages ago. The other about Certificate Transparency logging. Still going through the wiki pages to understand how that works, but would it really log an event such as Tailscale terminating the request, dumping the data, and re-encrypting them before sending us the request? Or is it possible for a bad actor to hide the logging?
I went all-in on Tailscale couple of years ago but slowly (and painfully) moving away from it.
It's a fantastic service but with a big flaw: its iOS app eats battery like crazy. I didn't know about it until I accidentally saw it one day: IIRC, it had consumed 20-25% battery averaged out over 10 days. (I used to keep it running in the background all the time, only to route DNS requests to pihole on a home server.) When I googled, it seems like a known problem on their forums for a long time.
At first it felt ‘magical’ being able to reach self-hosted services from my phone regardless of my location, but I quickly noticed the battery drain as well. I believe it has to do with an always-on ‘VPN’ and I don’t expect any improvements soon.
I’ve decided that Tailscale works perfect for all my computers (e.g., Raspberry Pi, Synology NAS, laptop and VPS), but not for my mobile devices. To mitigate this I use cloudflared on my VPS to route internet traffic over tailscale to any internal services that I often use on my phone.
Cloudflared has good options for securing a tunnel by using MFA methods, for example Google authentication.
For the rare occasion that I need to access something else I can always temporary join the Tailscale network from my phone.
It is not caused by an always-on VPN, but by Tailscale. Having e.g. plain Wireguard enabled 24/7 doesn't cause the battery drain or the other issues Tailscale on iOS has.
Unfortunately, Wireguard on iOS doesn't work very well with dyndns, as it doesn't re-resolve dns and thus silently loses the connection when public home IPs change.
What are you hosting from your phone? Personally, I think upcycled phones plugged into USB drives are the future of selfhosting, but the software's not there yet.
It also eats a lot of data, I recently had around 3gb eaten in a day just for maintaining the connection. On a metered mobile connection with 6gb/month a tad too much.
I‘d really like to use it, but in this state it’s really unusable for me.
For that they need a certificate. They have two options of obtaining that:
a) They request one from a CA. This will be logged in Certificate Transparency logs, and thus you could detect it by comparing the certs logged with the ones your local machine generated
b) they could have their software upload the certificate from your machine. That would need effort to detect (deeply inspecting the software and/or its traffic), but if a whiff of such a "feature" were to be found by anyone it couldn't really be explained away. (and aren't their clients app open-source? then at least it'd be reduced to source inspection and compiling yourself, which makes hiding stuff harder)
Thanks, you're right yeah, I've oversimplified things a bit.
Re: macOS and the Network Entitlements shenanigans: if I understand correctly, it is possible to just run tailscaled unsigned [1] via /dev/utun instead of Apple's APIs. Would it be possible to get this into the GUI so that if you want, you can compile it from source and don't have to do the Apple dance?
tailscaled isn't particularly stable on my machine though, so I guess I'll roll back to the closed source version. However, this could be a starting point for a Linux client!
If I CNAME my domain to my assigned ts.net then I'm guessing funnel won't work because of the SNI lookup will it? Is that on the radar? Right now I'm using cloudflared but if I can get rid of it in favor of this that would be one less daemon to worry about. Wouldn't be able to do esni though. sooo, how about a vanity .ts.net I get to choose?
It's very nice that it doesn't do TLS termination, but it seems to rely on SNI. That's cool, but it probably precludes SNI encryption in the future. Is there a plan around this? Maybe in the IPv6 world, it becomes a non-issue... But I wonder what comes first: the push for ESNI everywhere, or actually substantial IPv6 adoption.
(Edit; though come to think of it, in practice, if the IP address each host resolved to was unique, it wouldn't really matter very much from a security/privacy standpoint, so I guess it's probably not important...)
Author here. This shouldn't preclude doing us Encrypted ClientHello in the future. We control the DNS for *.ts.net so we can publish our public key for browsers/etc to encrypt the ClientHello to, if I'm remembering the latest encrypted SNI spec(s)?
Is there any updates on being able to use the IOS app for self hosted tailscale? Also what is the status of fully self hosted tailscale in general?
I just don't trust my VPN to be managed by a 3rd party like this. I'm willing to pay money even (though I don't have much) for hobby use - but I don't like the possibility of exposing network devices like this. To be honest I'm a bit surprised seemingly everyone else is.
In addition to your points, we over here also have our own reasons for self-hosting everything (for example, to protect ourselves from being cancelled at any moment for being forced into a citizenship you didn't ask for by being born at the wrong place).
Having an app that can setup its own completely secure networking no matter where it's run is game changing, as soon as I get into the alpha, the Cloudflare tunnel it's on right now is going down!
Exit nodes are usually still behind your firewall and have no open incoming ports. If you're willing to reconfigure your firewall to open incoming ports, you probably didn’t need Funnel in the first place.
What do you mean by "trust", what's your threat model? Tailscale does way, way more than just facilitate key exchange. If tailscale.com goes down or rogue, you're still in a pickle even with PSKs; just because there's a Wireguard under the hood, doesn't mean you can swap an API endpoint and continue as if nothing happened.
Even with self-provided PSKs, you're going for an (IMHO) pretty poor trade-off; keys, certificates, etc should be regularly rotated, that's a chore that's best left automated. At that point, why not just set up Wireguard yourself?
If you have legitimate concerns, you should be using Headscale[1] (or even plain Wireguard) from day 1. Otherwise - personally I find the current threat model very reasonable, it's in no way worse than trusting any other VPN provider, and they're keeping a pretty big chunk of their code base open for auditing.
Tailscale is really, really doing a good job of taking all of the different secure remote networking tasks that have always been possible but self-managed and tedious, and wrapping it all in an automated, friendly interface.
Every time they come out with a new product, I read about it and think "Wow, that's really useful. That would be such a pain for me to roll by hand."
Everything they do was already possible with open source tooling, but they make it accessible. It reminds me of when Dropbox first came out.
I really want to like Tailscale but I couldn't easily find a way for it to work on a jumpbox running Ubuntu 14.04 that I don't have root access to.
Anyone have a way to install the client on a non-rooted box? I couldn't find anything in the docs about the assumption that it requires root or not, other than it being implied by their install steps using system package managers.
Something else interesting they're doing is their tsnet package, which lets you join your process to the tailnet and bind tcp listeners/connect to TCP services via their tailnet IP or subnet.
I'm writing some stuff using this at the moment, but I also just saw https://github.com/tailscale/golink which does the same thing: a single binary that runs a link shortener that joins itself to your tailnet.
tl;dr: don't run your service on a machine then join that to tailnet, directly bind your service to an in-memory tailnet client
Its very cool, some thoughts:
* They have support in there to get an API client that auths using your nodekey - not sure what endpoints it can hit but makes for some interesting cases like updating ACLs maybe
* You can use it as an exit node/subnet router too, but it requires some hacking
* Equally there UDP support lurking in their library you can wire up too
This is yet another amazing cherry-on-top of an amazing product. I switched Internet providers a couple months ago and was horrified to find that I was now behind a CGNAT. I went looking for a solution to reach my home lab on the go and found Tailscale. Tailscale solved that issue - and I actually had the epiphany that if I couldn't reach my home network without it then nobody else could either. So, maybe the CGNAT is actually a big security benefit in that way - but I digress.
I originally was thinking of it like a network-level VPN but realised if I installed it on everything individually it would give me DNS and HTTPS certs for all the machines in my home lab - that work from anywhere as long as my laptop was connected to Tailscale. That is something I always wanted to do with let's encrypt but never got around to. And now this!
It has really inspired my imagination. I've been running labs teaching 5-15 people k8s and k8s security out of AWS but this means I might be able to just run a bunch of VMs in my home lab all with Tailscale loaded and point people easily at them all. Maybe with code-server (VS code in a browser) on them to give them a browser-based terminal. And that is just one possible usecase...
Thank you Tailscale people - it is such a great product that has exceeded all my expectations!
A "dark webhook" in this case just means sending data from a public server/service like a GitHub action (or even an AWS Lambda) to a server that doesn't have a port open on the internet.
For example, I get a message to my Mattermost client everytime someone pushes/commits/etc. All those messages are sent to the Mattermost server over a webhook that is configured on an overlay network (in our case, using OpenZiti). There's no firewall rule allowing the traffic to the Mattermost server, it's "dark" to the internet.
Sounds promising! I recently got gigabit fiber through Sonic, who does not offer static IP addresses. I did read the article but it is pretty deep in to some stuff I am not that familiar with. Can anyone clarify, if I own some domain name, can I host something on my raspberry pi or whatever and use funnel with my domain so that someone can visit my domain and get to that hosted thing? And how would this compare to dynamic DNS approaches? Thanks!
No, you can't use your own domain, this specifically works with the ts.net subdomains. You could use it to host your service on "node.tailnet.ts.net", if you're ok with that naming. Otherwise, you'll need to set up a VPS with tailscale on it, and your own domain, and have it proxy similar to how Funnel works.
I assume there are going to be some egress charges for Funnel. Unlike normal tailscale, Funnel has a high bandwidth component to it for hosting high bandwidth services.
That's a good point. I have a super simple status page I have wanted to make that would be externally accessible. I don't care what the URL is for that one. I guess this would be a great use for that! That one is behind a Starlink terminal so again, no static IP available there.
Anything stopping us from just creating a CNAME to our tailnet domain and registering certs for it instead of whatever.ts.net? This seems like it should work in my head...
It would be great if Tailscale adds this, but there are lots of services that provide this functionality if you need it today, including Cloudflare Tunnel.
From how I understood the article, they don't do TLS termination but they do SNI snooping to figure out how to route it? So if they don't have all of the infrastructure in place to map the SNI for your CNAME to your Tailscale network, that wouldn't work?
The next step is being able to send a redirect to the client's tailscale with the public key of the destination node, so it can establish an anonymous connection directly with the node running the service, using some sort of Funnel ACL (default deny).
Also, I've been told that it's currently bandwidth limited, with the future holding maybe variable bandwidth limits and bandwidth charges. So a dynamic DNS pointing at your gigabit home connection is likely going to be a better solution than funnel.
I wonder why that's the case. If you own a DNS name, you can set NS record to point to magic-blah.tailscale.net, so they'll have a chance to geo-locate you and return IP address of the closest VM.
We left them because of this. Long overdue move away from Gmail Gsuite resulted in TS costs for our dozen user team at least doubling; the overhead involved in managing the overlay network acls was also getting out of hand too.
Ever since I started using Tailscale a year or two ago, I haven't had to think much about networking for my self-hosted servers at home. It all _just works_. Tailscale is a great example of software solving a real problem in a user friendly way.
Are you trying to share home assistant with everyone on the internet? If yes, then yes.
You probably don't want that, though, so vanilla Tailscale would let only you securely access your home assistant server from anywhere, as long as you install the Tailscale VPN app.
If I understand correctly, this still won’t let me achieve my dream of hosting my personal email on my home computer, because smtp connections typically start out as plaintext and then negotiate up to TLS within the protocol itself.
One nice way this could be improved is to add support for a wildcard dns record (e.g. *.host.tailnet.ts.net) so you are not limited to a single domain per host.
In addition to the "show your dev work to the CEO" use case, tunnels (ahem, "funnels") like this are useful when you're building functionality that requires pointing webhooks at your devlocal environment.
e.g., if you're building an integration with any of the myriad of SaaS tools that fire webhooks, you can test in devlocal and have a URL of whatever.tunnel.com.
There are tools like ngrok and localtunnel that exist to do just this. I'm looking forward to replacing those with TS Funnels.
Or more realistically, if you are in web dev and wanted to show your work on your local machine - say when the CEO wants to 'check in and see how its going', you can just send them a link on slack and they can play with what your developing on.
can't the CEO just use the Tailscale network to access it privately?
I just don't personally see a use case -- if I need someone to access the content, the box it is on is already easy to publicly access. if it's not, the only people who need to see it are on the Tailscale network
edit: I think I'm starting to see it. it's not for long term installations. I just feel like, why not just open the box to the internet at that point.
it makes more sense for the edge case(s) that might address friction with adoption. interim solutions, or temporary connections
edit2: I can also see how this might be useful for people living under an oppressive government
Do they want you to host a public website? AFAICT all the inbound traffic to Funnel has to go through servers they operate, and historically Tailscale has tried to minimize the traffic they handle that way.
They explicitly mention selfhosting services like Mastodon in the article, so have high hopes that they will support use cases like this moving forward.
the more I sit with it the more I can see different ways it might be valuable. even if it's just to simplify setup. the use cases just didn't really jump out to me this time, even as a happy Tailscale user
As someone who's currently using ZT (both personally and at work), and evaluating switching over to TS...
ZT is absolutely brilliant, but it lacks polish in a few places, that sometimes turn out to be critical. I could probably do a side-by-side comparison table, and ZT would still win in roughly 50% of the cases, but the places where it's losing are causing tiny bits of friction.
ZT flow rules are approximately just as powerful as TS ACLs, but TS has an integrated editor that does formatting, validation, inserts hints and snippets for you, etc. TS ACLs also have test rules that allow you to create assertions, about what must (or must not) have access to what, increasing my confidence when making bigger changes.
TS allows you to tag assets, making complex ACLs easier to reason about.
TS has MagicDNS; with ZT we had to hack something together using their API and a bit of Terraform, and it needs to be rerun every time a node joins/leaves. I could probably write a tiny custom authoritative DNS resolver that talks to ZT API, but TS has it out of the box.
We've effectively soft-bricked a ZT node behind an NAT during a routine dist-upgrade, because dpkg stopped to ask a question, in between the old ZT client version going down, and the new one coming back up. That costed us having to go on site in person to service the device (luckily just that one, and not the whole fleet - but we should've picked a canary node with secondary means of access, lessons learned).
And to be fair, TS is also far from perfect: it's currently allowing one node to participate in only one tailnet, and switching tailnets requires re-authentication in the web browser. This is incredibly annoying in any BYOD scenario, if you want to have separate personal and work tailnets. ZT is infinitely more flexible here: any device can just join or leave networks at will, and it's up to the network admin to authorize it (once).
The list goes on, I really like both tools but the small differences keep adding up in TS' favor.
Isn't the performance also a difference? From what I understand, Tailscale uses wireguard, and ZT uses some other L2 tech. I haven't had the chance to personally compare, but it seems like that could be an order of magnitude difference in bandwidth between the two options.
It's worth noting that Tailscale runs WireGuard in userspace, which negates many (all?) of the performance benefits. I wonder if it will ever be possible for nonroot users to create WireGuard devices. They're just so useful.
Out of curiosity, what kinds of performance benefits are kernelspace-exclusive, in terms of VPN apps? I've seen arguments go both ways for different applications, with full usermode TCP/IP stacks, unikernels, even running apps directly on the NIC (snabb switch); historically there were also efforts like khttpd that ultimately failed.
I don't know much about what where specific gains come from. Generally avoiding copies and syscalls go a long way. I suspect in many cases WireGuard performance isn't going to be your bottleneck, but in those cases you're still reducing power consumption.
We're neither latency- nor bandwidth-sensitive, and both services are overall quite reliable, so I haven't actually paid much attention to that aspect. But there are plenty of war stories of people using ZT or TS to do absolutely bonkers stuff, like zero-downtime migrations between cloud providers, with a write-heavy DB in there.
I wholeheartedly recommend doing your own benchmarks! Everyone's workload will be slightly different, ultimately you should pick what works best for you.
I'll definitely have to do my own benchmarks! It's been weird how infrequently VPN tech is benchmarked. It's not easy to dig up hard numbers on bandwidth.
For this particular feature I like my own home rolled solution where I automated the process of spinning up a tiny ec2 server with an nginx proxy to a reverse ssh tunnel that goes back to my local machine. It gets a letsencrypt cert and modifies the route53 dns. All managed with a cli command and config file.
The other benefit is that it is completely unthrottled, which is particularly useful if your debugging some stupidly large react/vue website.
Im working on a feature that will enable basic-auth in front of it also, so at least a password would be required to send traffic to your local machine, if you wanted. Good for the "share your work" sort of scenarios.
Unfortunately none of those solutions are self hosted AND automated. They all seem to either use their own ifra, which will be slow & throttled, or require setting up a server and manually editing DNS.
I may try to get my own project added to that list though, so thanks for sharing!
Well, yeah - For a minimum of $20/month just for the personal license!?! and THEN I need to still pay for my own infrastructure, AND run all the setup, etc.
mine is free and as easy as 1,2,3:
npm i -g tolocal
tolocal config -> answer the 3-4 questions about domain, vpc, etc.
tolocal apply. -> tunnel will come up to my own custom domain.
Because it’s not the basic Wireguard part itself that’s hard, it’s the synchronized keys, NAT busting, ACLs, and other things on top of it that are. Tailscale has made it incredibly easy to deploy and use confidently and that’s why people are happy.
Given the immense trust I have for the names behind Tailscale, sure. I don’t run my own data centers, either, so it is a sound business decision to offload this too.
For exactly the same reason that Dropbox was just "rsync with a shiny UI" and was very popular.
It's less about the UI and more that Tailscale takes care of setting up wireguard connections between every machine that needs them, along with the authentication and access control. All of which ends up with a simple network between your devices that doesn't have to worry about physical location and intervening network infrastructure.
Usually people are pretty critical/cynical of sending sensitive data to a closed source third party server, no matter how strong their claims of 'being the good guys' are (eg see Telegram).
But somehow we're all meant to be happy giving full control of our entire network to a commercial company running a closed source command and control server?