https://www.indiehackers.com/ is a great source for this sort of question. They provide various filtering mechanisms as well such as revenue, number of employees, business model, location, etc.

https://tv.youtube.com/ is also down

Pitch it to Amazon as a proper monitor for their AWS Cloud.

I'm on a high floor of a highrise in Uptown, Dallas. The building was shaking and swaying hard enough to wake me from sleep.

Tools such as the percona toolkit (https://www.percona.com/software/mysql-tools/percona-toolkit) provide the ability to perform safe and performant online alters for MySQL. Behind the scenes it actually creates a new table with the new schema and utilizes triggers to apply writes to the new table as the original data is being copied. Once it completes the table is renamed and the triggers are removed.

Percona also recommends using this tool to safely perform alters for any Galera-based replication product (Percona XtraDB Cluster, MariaDB Galera Cluster, etc.)

I have used this tool many times in production, and in accordance with advice about avoiding foreign keys.

It only failed catastrophically, causing a critical production incident, twice.

Each time took dozens of engineer hours to vet in advance, thus costing thousands of dollars.

Online DDL changes and indexing with pg cost us... basically nothing, and never caused downtime.

My and Pg each have their place... but if you want to modify large tables, MySQL is almost certainly the wrong tool for the job.

While this is true, i don't personally think Percona tools alone makes these changes 'safe'. You can create scenarios where performance is so degraded that you introduce replication delay or service interruptions (and yes the tools have ways to work around that, but it's a significant challenge to tune this correctly on production), data can be lost during the table switch over, and the complication of managing a product / database with a table in an extended migration is very difficult.

This space is far from done, until we can change data structures on the fly...we're going to find there to be a constant struggle between schema and schema-less and neither side will be right or wrong.

Solution: Quantum states; We suspect every possible scenario is happening already, so we should just migrate our database to the appropriate quantum state, where the data is in the structure we desire. Sounds insane, it probably is.

Your absolutely correct that it is not risk free, however, if you do not use foreign keys (which no huge scale mysql install like schemaless does) and you know what your doing (read: calculating the expected load, adding throttles and sanity checks, backups, etc) online alters are extremely feasible with mysql and folks have been doing them for years.

I know first hand shops like FB use a similar method in production.

Also, big note, the whole point of "schemaless" (And other things, like fb's friend feed architecture) is that you don't make schema changes.

online schema change is absolutely riddled with crippling bugs if you use foreign keys. Just a heads up, we've had to basically take everything it does and make an in-house version. The idea and execution are great, when it actually works.

Foreign keys are a major problem in MySQL if you do lots of writes, especially bulk writes. As is auto-increment (table-lock for load data infile, yay!).

We allocate our own IDs, and disable foreign keys for bulk loading.

I believe he now works with DataStax on their DSE Solr product.

Goagain by Richard Crowley is a great package that we are using for graceful restarts: https://github.com/rcrowley/goagain

EDIT: added author

I found that it corrupts the FD by using this call:

https://github.com/rcrowley/goagain/blob/master/goagain.go#L...

Not sure why it happens though, but it led to all sorts of strange intermittent issues with broken connections.

Once I replaced this logic with passing files using GetFile().Fd() instead it started working fine, so beware of this. I still wonder why it happens though.

Thanks for this.

Were you able to publish your changes either on a fork or in a PR?

I ended up using different library:

https://github.com/gwatts/manners

But the code that extracts the fd without using reflection and access to the private properties is here:

https://github.com/mailgun/manners/blob/master/listener.go#L...

I think it should be fairly easy to port it to Richard's implementation

CVE-2014-6271 was posted on the AWS Security Bulletin earlier today:

http://aws.amazon.com/security/security-bulletins/CVE_2014_6...

My guess is that they cannot rely on all users to perform these updates.

can someone explain the downvotes?

Already confirmed by AWS its not related. Confirmed in this thread its not related. That issue is an OS level problem, AWS won't touch your OS.

On a related note, this book (http://www.treesmapsandtheorems.com/) is excellent.

The author approaches communication from the principle of Gestalt psychology and really helps one understand how our minds communicate.

From the description it sounds as if this provides synchronous replication. How does this work in a WAN environment?

Well, "synchronous replication" is used in a couple of ways here.

Tables can be designated as replicated or distributed (sharded). Replicated tables are typically fairly static. These are handled synchronously in the cluster on every datanode where the table resides. Actually, it first applies to a designated "primary" node, and upon completion, it will execute on the other nodes. The reason for this is to reduce the chance for deadlocks; if it succeeds on the primary, it has obtained all of the needed locks and we can be sure we can then get the locks on the other nodes.

In addition, the term synchronous replication is also used as in PostgreSQL's replication, but in Postgres-XL for a datanode having a standby replica, synchronously. It is a warm, non-queryable standby.

With regards to a WAN environment, Postgres-XL is not designed for that because of the latency, and for handling long network outages. If there is enough interest, that could be an area to enhance in the future, but consistency may be compromised.

I didn't get replication out of the description. It sounded more like distributed storage and load. Notice they say "warm-standby" for data nodes. I can only think of one reason anybody would resort to STONITH for something like this.