Ochan

Ochan 1.0.0 released. Check out the demo site, its already updated.

Next scale based vision; breaking off services to individual machines/databases.

We need to look at how we would tune the database to be more blob centric versus the small data of the categories, threads, and posts.. perhaps one vision would be to break off the big blobs to a high memory machine; and maybe even break off the thumbnail generation to those machines as well.

Just a thought.. seeing as how .26 isn’t ready to be cut, we might as well rearch. some more before we do a 1.0.. and then maybe walk away from this project.

Yay for having physical machines to test with. Also.. yay for replication actually being a 2 hour job to introduce. Now, to get elections to happen when nodes fail.. more reading is required.

Replication work has started.. I have some doodles to do in my notebook before things go into full on real world test mode. Expect the ‘ultimate solution’ to be a mixed bag of the sharding, replication, distributed ehcache, and some secondary squid caching for bonus awesomness. The big question I’m going to have to address is how to wrangle about 50-100 servers for a hard core load test… with failures.

oh the pain… the pain that will be in store for me. (and the learning opportunity for doing opscode and cfengine and the like)

I’ve just started thinking through some replication strategies.. to make up for berkeleydb java edition not having it. Ochan still needs high availability.. and I think I’ve figured out a way for in-place replication…

I wont release it until I have auto-regeneration scripts to show it off.

Example:

Web tiers A, B, and C

connected to Backends X and Z… with a hot spare server named ‘Joe’.

And a sync system S.

If X fails, Joe should become the new X. But, also, Joe should be able to be either X or Z.. (so, it should be unsharded). And Joe can be a backup for S as well.

We already have HA in the web tier; A,B, and C can fail and no one cares. But, having a replication group is kinda useless unless ‘something’ can identify the new middle tier replacing the failed one, and reconfiguring the webtier to start calling it.

Service level sharding.

Ochan is now capable of a scaleable deployment model in the development version. I’m going to do some more testing before making this the 1.0 version of Ochan. But, this isnt the end of the road of scalability and the pursuit of the most flexible image board software on earth.

How does it work?

The standard Ochan release contains everything you need, as usual. But, now, you can deploy the application as:

1. The synchro service (the master service commander)
2. The middle tier with attached/embedded database
3. The web tier

The basic deployment just works with it all configured to run on its own.

But, you can deploy all tiers seperately. The synchro service can be load balanced for high availability. The web tier can be load balanced for increased traffic handling and high availability. And the middle tier can go as big as you want (but, like a raid 5.. if 1 fails, the whole thing fails…. (unless you load multiplex each middle tier deployment with a backup (raid 10)).

And, well, this is going to have to be the solution until berkeleydb java edition grows up and adds replication.. and then.. the synchro goes away, and the web tier can be its own distributed middle tier.

I am going to build up some tools for re-sharding an existing database.. as, well, everytime you contract or expand the shard, it’ll require some effort to redistribute the data.