I am glad they did this, as a user who already left the big G earlier this year. Its wasn't a big priority though because of IMAP and ICS. I already got my data out. If anything, I hope this helps promote the MBOX format for quickly moving mail around vs. error prone IMAP transitions.
It's pretty cool do download an archive though, instead of going through IMAP, for archival purposes that is. I have a lot of email in my GMail, going back to 2005 I think. I'm afraid of accidentally deleting it, plus I also use other email clients, like Thunderbird and I'm always triple-checking settings to ensure it doesn't delete old emails. Having the option of doing a dump from time to time, for storage, is really cool.
Check out "offlineimap" (or similar utilities) that sync your Gmail to your local machine over IMAP. Run that via cron and can back up your mail like normal.
To play devil's advocate - you willingly gave it to them, with almost zero contractual expectations. In a technical sense it is their stuff, and they have graciously implemented a feature allowing you to retrieve a copy of it.
I definitely agree that it's the "right" thing to do, but this kind of comment speaks of entitlement and lack of personal responsibility. They're offering a service, they don't have any inherent obligation, and you can choose not to use it.
Absolutely not. The implicit agreement is that they provide a service to hold, manage, and use your stuff, and you give up a certain amount of privacy by allowing them to see it - and they get to advertise at you while you use the service, and maybe make money in other ways off your information.
So in other words, it really is your stuff. Imagine if they locked your accounts from you out of malice - it's legal gray area, since I don't believe there have been any court cases, but I'm pretty sure the result would be obvious. You would easily be able to sue them to get back your access to it.
Why would you have the expectation of an implicit agreement? There is no implicit agreement, that's an unfounded assumption by the user.
I imagine they could completely shut down all unpaid gmail access at any moment with no legal implications (though I haven't read the fine print, and in that extreme case the courts would probably step in just to keep the country running smoothly).
What precedent is there to support your hypothetical lawsuit? I kind of disagree, they would just get a lot of negative PR.
If Google simply shut down Gmail, their stockholders would pretty much fire the entire leadership of the company. It would be professional suicide to approve something like that.
There would certainly be a change of leadership due to the change in the business that shutting down gmail would cause. The PR hit would be huge, and I'd WAG that majority rule would not be enough to keep them in place.
Implementing exporting features takes time and money. If such things would come for free, then they would have an obligation.
Yes, it's your stuff and I don't agree with the parent. But you're not necessarily entitled to features for an easy export of that stuff. That's the kind of agreement that has to be explicit in order for you to feel entitled.
Anyway, if you want fair treatment and to feel like a customer and not a product, you should pay for using it. I'm on Google Apps, I have my own domain and email address, I can always move, I don't get adds, I can use the Exchange protocol, I have more options for importing/exporting and the email support has been very responsive for me. From what I hear Office 365 is a solid option lately and there's also FastMail.fm and others.
And btw, allowing them to advertise in your Inbox does not make you the customer, you're just a user of free stuff. And free stuff is great, but not when you depend on it (unless it's open-source :))
I have ambivalent feelings too but more in a gracious sense. I have long hold the philosophy that data should be treated as a consumer's personal property. Companies have reasonable rights to refuse to serve but they shouldn't be given authority to confiscate consumer's personal property and close their accounts unless warranted by the law of the land.
A person who lived by this "technical" principle would likely find that their friends no longer wanted to loan them anything. After all, there is no "inherent obligation" for you to return anything, right?
It's only because data does not have legal status as property. It doesn't keep people from feeling a similar degree of ownership over the data they store with Google et al, though, and I think it points to a democratic desire for these data-retrieval acts and implementations to be more than goodwill. The third-party doctrine is only a convention, after all.
Until you get the "lockdown in sector 4" error message and have to sit in the timeout corner to start again. It took me over 48 hours to actually download all my mail out of gmail. (about two years ago.)
Really? Every email application I've used in the past few years asks for your name / email address / password on the first wizard-page. They automatically detect gmail, hook everything up, and start downloading without any further steps.
I've never had that problem. I have plenty of other annoyances with Thunderbird (so much that I'm paying for what's essentially a custom UI), but it's usually stable. It hasn't complained when I've hooked up a half-dozen accounts, collectively about a half-million emails, and it did everything reasonably quickly.
Hmmm... not seeing the same behavior today. There have been upgrades in the meantime, of course, but I doubt that this was addressed. Must have been something off that day. It did run off and start downloading the header of every single message in my GMail account, which I would rather it didn't as I really only care about the mail that I have received recently (I only use Thunderbird for encrypted emails but the web interface otherwise). That is still annoying, but since I don't want to bad mouth people with false information:
Let the record show that Thunderbird no longer becomes unresponsive for me when accessing my GMail account, must have been a glitch on that day when I first set up GMail in Thunderbird.
haha, I don't mean to imply that it's impossible that it did that, just that it shouldn't be normal :) usually bugs like that are fixed pretty quickly, especially for something as widely-used as gmail.
I think this is fairly obvious but there's a clear difference between "data belonging to a user" and "data about a user". The first is what they are talking about, because this is how vendor lock-in occurs.
That first link is hilarious. I looked at my 'interests' based on my searches and apparently I really enjoy Athletics and Chevrolet. Weird considering I don't like either haha
For anyone looking to an alternative method, there is a great open source Python script called Bagoma that incrementally backs up your Gmail account: http://bagoma.sourceforge.net/ . Each email is stored as an individual text file rather than the mbox format. I have backed up my account with it every few days for a couple of years and it works great. It will even restore your account to Gmail if you need to switch accounts or accidentally delete messages.
An alternative to this would be fetchmail and maildrop. The MTA (fetchmail) downloads the messages periodically and delivers them to the MDA (maildrop) which can store them in Maildir format (each mail is an individual file). Fetchmail actually has weird bugs with some mails, so I made my own version in Perl, but there's other options out there.
2. fetchlimit was set to be less than the number of messages in the remote mailbox, and
3. The remote mailbox stayed above this limit,
then you would find yourself in a state where fetchmail would say that it had fetched everything while there were unread messages that it would never look at.
Looking for a bug report just now, I didn't see anything that matched this.
I don't know about lost messages, but it used to bomb out on spam, but still deliver the messages up to the spam. So I would wake up with 5,000 duplicate messages and a lot that were never gotten, and i'd have to use a client to delete whatever seemed like the "bad" message until fetchmail could continue delivering mail and finally delete the old ones.
To solve it I did what any bored Perl programmer would do - write my own fetchmail program in a couple hours in Perl. It solved my issues and i've been using it ever since. Years later it took me about an hour to add SSL IMAP support so I could use it on Gmail and Exchange.
FWIW I needed to do this to back up business emails when I closed down my last shop. At the time, all the open source projects I found didn't manage to correctly download ALL messages. I was finally able to do it by doing an HTML inject, and using JS to load each email iteratively.
2-3 years ago, and I don't remember, unfortunately. I just wanted to point out that it could be done through simple javascript if those viewing this thread are unhappy with any of the libraries mentioned.
Mbox, eh? Can anyone confirm that they've implemented this properly, particularly when it comes to escaping lines in message bodies that start with "From "? Failure to properly escape such lines was an infamous bug with Mail.app's mbox exporter[1].
How is mbox a broken format? It's a bad idea to use it in a multi-writer environment, and it's horrible because there are competing versions of it. But for plain storage, I thought it worked quite well.
Any line following a blank line and beginning with the four characters "From" marks the start of a new message. The BSD way of handling such lines in the body of a message was to prefix them with ">", which is a non-reversible workaround and is in the uuencode target charset.
Prefixing every line in the message body with a tab character would allow reversing and not interfere with uuencode/base64&c.
I've got mbsync set up to periodically dump my Gmail into a local Maildir archive. Works great, and it was a (relative) breeze to integrate with my old mail archives reaching back to the late 90's.
Unfortunately this requires you to be able to log in. I had an account (with the same name I use here) mysteriously stop being able to log in about 6 years ago. I lost all of my email, contacts, and calendars.
I was never able to find out why the account was shut down, I'm guessing it was a ToS violation for something I did several months before I got banned (made 100+ gmail accounts with my invites, didn't use them for nefarious purposes though.) Just one day I couldn't log in and got an ambiguous page about account deactivation. No amount of emails or form filling ever earned a response from Google. That was it. Blam. Gone. Ever since then, I have not trusted Google with any data that I care about.
It would be nice if this tool worked for deactivated accounts.
No, I would like for this tool to work with deactivated accounts so that data can be retrieved in the event of Google deciding to deactivate the account.
You could achieve the same goal if they let you reactivate the account and then retrieve your data (either with this tool or with IMAP).
I suspect that the tool works by taking the data from the same storage that is used for active accounts. So, assuming that your data is still there, they would have two options: a) write another tool that exports data from some backup or b) first restore the data and then export it.
In any case I don't find your feature request to be relevant to this topic. Sorry for nitpicking :-)
Why would Google keep a copy of your mail if your account was deactivated for 6 years? Isn't that asking a bit much considering you were the one that broke the ToS?
My hope that the tool is activated for deactivated accounts is more for people that suffer the same fate today - if they lose all access to their account, that they at least be able to get their data/life back.
Does anyone thing that Google isn't keeping a copy of every bit of data they have on everybody that has ever used any of their systems? I cannot imagine they would ever throw data away. Even if they restrict the end user from it doesn't mean that they destroy the data for their own use.
support@github simply doesn't reply. Apparently, creating https://github.com/excelsiores/geek-misandry was enough to get me onto their "block and never reply to this user" list.
Sad times. Wish they provided some way to export data before kicking off users like this.
At least git is distributed so you should have copies of anything you cloned :)
Sadly, this is the way of tech companies though - skimp on real customer service & replace it with one sided FAQ's and unanswered forms/emails. The only way out of these situations is to get into the public eye & force a response on their part.
I was once locked out of Google Groups. No idea why, I used it very little.
My solution was to grab a free Google Apps of Business trial. Now I had direct access to support and could get unblocked within a day.
If you use a native email client and this client saves every mail on disk, you have a chance to have it backed up with everything else. It's still far to be the generic scenario, I think, but it should raise a bit the number of people having their gmail backed up.
I use IMAP clients on all my computers to access gmail so they inherently have a local backup. Then at the beginning of each month I go to Google Takeout - https://www.google.com/settings/takeout - to download all my Google stuff and add it to a backup that ends up in Dropbox. (I should also add that I run my own domain and email so gmail is just a backup account.)
However on checking takeout it turns out the one thing they don't provide is gmail! 8 of the 15 items backed up are related to Google+ though.
This is great news. Unfortunately downloading my Gmail content isn't listed as an option. I hope this isn't another case of Google App accounts not getting new features.
We are rolling this feature out slowly, right now only about 1% of people have access to Takeout's of GMail. But we hope to increase it to 100%, including Google Apps accounts (if your administrator hasn't blocked it) over the next couple weeks.
OK, my archive is being collected as of now. It was so easy, with a direct link to the takeout, with Gmail preselected, and a one click operation - I almost wonder what the catch is :)
I suspect it might have been a bit of a battle to push this through the top management. Anyhow, whoever did this - thank you.
Here is the simplest solution I found to taking a backup of your gmail. The only glitch is that the mirror account will of course be a perfect image only if you set up the forwarding from day one :-(
I had to use gmvault (gmvault.org/) to make a copy of my gmail; it was quite doable.
"Create some backup accounts — This one is so obvious I’m always surprised to hear that not everyone does it. Gmail is free and lets you have an unlimited number accounts… so open a second one. Mine is just my usual account with .bak appended to the username. Once that’s set up, login to your primary account and either enable forwarding for all messages (Settings >> Forwarding and POP/IMAP) or create filters to only forward the messages you really care about. Then just send those messages on to the backup account.
I use my main Gmail account to manage five e-mail addresses, so I used the filters method and only back up a couple of them (the others are already backed up on the original domain’s mail server). I also have the forwarding set up to pass messages along to a Yahoo Mail account, an account on my own server and even a good old Hotmail account (or Windows Live Mail as it’s known these days).
This setup is like a good investment portfolio, think of it as e-mail diversification — your’re hedging your bets by spreading your mail around to several servers. If something goes wrong on one account, just login to another.
I don’t login to the other accounts much, so they aren’t as carefully filtered, organized and de-spammed as my main Gmail account, but at least my mail is there"
Thanks for sharing. In the case of GMVault, how "important" do you think your email needs to be for you to use something like that, as opposed to using Google's solution when it comes out? Or asked in a different way, what do you feel GMVault could offer which regularly archiving emails using the Google solution can't?
I used GMVault only because Google's solution was not out there at that time. However, as others have mentioned, GMVault works on an incremental basis after the first download, while the Google solution seems to be a snapshot-only service (not 100% sure yet).
By the way, my first download with GMVault (eml format, each email separately gzipped up because I did not read the command line options properly) was : 22900 mails which took up 2 GB of disk space.
You can automate gmvault to snag everything and then just do daily/weekly syncs instead of manually having to remember to go to a website every once in a while and download everything each time.
I've been using Thunderbird (IMAP) to keep a local copy of my gmail for ages. It works very well, and as a bonus, you can sort by field and get more than 20 responses per page to a search.
The problem I have with this download a copy approach is that it's not a great backup (not automatic, not easy to find individual things to restore, etc).
For that, I use CloudPull from http://goldenhillsoftware.com and it's by far the best Google backup tool I've found. I've tried gmvault, fetchmail, offlineimap, and pretty much every thing in between.
CloudPull supports Gmail, Google Contacts, Google Calendar, and Google Drive/Docs. The app backs up Google accounts in the background and all the data is available in standard file formats.
Anyway, I don't think enough people know about it. If you want a useful automatic backup of your Google life, give it a whirl. It's one of my favorite pieces of software.
"Got Your Back" https://code.google.com/p/got-your-back/ is a pretty awesome python script that downloads/backs-up all your gmail to your hard drive. I've used it to backup 6 years of work email before leaving the job - then created a new gmail account and used GYB to push all those emails back onto an archive account (for easy indexing just in case I ever need to reference an old email).
There is always IMAP access. I had no trouble downloading everything via IMAP every time I wanted a copy on a new machine (and a random person on the internet would expect no significant trouble when setting up a new mail client, so making this difficult seems impossible).
Not impossible, though I'm guessing gmail is used widely enough that it would be a PR nightmare if they did this, much worse that the greader fiasco. They're probably forced to keep gmail alive, whether they want to or not.
I've been using gmail exclusively on my iphone for years with no ads. They've lost whole dollars providing this service to me.
But they want to make money off me, not lose me as a potential revenue.
So my bet is this is in preparation for showing 'sponsored' emails in your imap/pop3. "If you don't like it, just download all your emails and use another service" will be a talking point to suppress outrage, knowing that few people will actually go to the trouble of changing their email address. Fanboys will lap it up.
I recently migrated away from gmail and before I left took a back of "All mail" up using mbsync ( http://isync.sourceforge.net/mbsync.html) which was easy to setup and quick to run. As a bonus this gives you your mail in Maildir rather than mbox.
Virtually everything supports MBOX (which, modulo a few gotchas, is just a concatenated file of the text-formatted RFC822 messages as they went over the wire). Most mail clients support it. Many mail servers do to, so for example you could drop this into /var/spool/mail on a AWS instance, run a default dovecot install, and hit the server with any IMAP client.
mbox is a standard email storage format. It's not ideal for a live email system, but not bad for archiving or for import/export purposes.
You can import mbox into variety of desktop email clients and, using IMAP, upload them to web-based UIs.
Here's how to move off Gmail to Rackspace (not free) email, I've done it last weekend [1]:
- Enable IMAP access in Gmail
- Add Gmail account to Apple's Mail.app
- Wait for Mail app to sync.
- You have a full copy of GMail archive on your computer now
- Select all messages in Mail app, and select Message/Copy To
- Pick Rackspace inbox and click "OK"
- Delete all your email messages in Gmail, purge Trash.
- Done.
I also exported Gmail labels into separate mbox files just in case.
I've done similar things a variety of times, the download -> copy -> paste routine is pretty simple. So far I've never had a problem with it (even moving 100k or so).
Mac Mail.app can import mbox files but the default storage is emlx if I remember correctly for Spotlight indexing.
Thunderbird should also import mbox
I am a bit paranoid about my Gmail account so I use the Mail.app to download the emails and then I back it up. If my account gets locked for whatever reason, 'Download Data' option is not going to exist. So better to backup through other means as well.
Many desktop mail clients support mbox format. Thunderbird uses mbox. There's also a script (I'm sure there are many scripts that do this) that take mbox files and upload the mail to an IMAP server. (http://imap-upload.sourceforge.net/)
MBOX, cool! I wonder if you could do this to speed up the full download of your Gmail inbox into Mail.app (etc) rather than letting Mail download each message, one at a time.
It will be interesting to see how many people that have been demanding this for years actually make the leap!
Not really imo…they are an automated, set-it-and-forget-it backup service. Not quite the same thing for me to have to remember to go in occasionally and download a full copy of my data versus having a service do it for me once or multiple times a day. Especially for businesses who want to manage multiple accounts and have a centralized backup system.
One "restore" preference people have is account migration. Take all your email from one gmail account and import it into another.
Plus your response makes my mind boggle. People consider their gmail way too solid. There's lots of posts online of people who have lost their accounts completely.
Exporting data via Google Takeout is supported for 15 (and counting) products, including Youtube, Google+, Drive, and more. Google made a commitment to make it easy to extract your data, and is executing on that. It has nothing to do with shutting services down.
At sodding last. Previously they only allowed IMAP interface, but their IMAP was fucked-up and broken. Most of the development work on offlineimap was compensating for Google's blithe disregard for piffling details like the RFC.
See the reply from bwillard above: it has only been rolled out to 1% of accounts and will be ramped up to 100% over the next few weeks (assuming all goes well)
Glad to hear that the Data Liberation Front is alive and well: http://www.dataliberation.org/
From 2006: "We would never trap user data," he [Schmidt] said.
[1] http://www.zdnet.com/blog/btl/google-ceo-eric-schmidt-we-wou...