Adam Wiggins says that SQL Databases Don’t Scale.   Some of the comments there mention Oracle RAC being able to scale quite well, but I do not know if RAC still has a scaling limit, albeit higher.   Maybe Oracle’s RAC limit is effectively infinity (much like no one will need more than 640K ram), in which case RAC solves our problems, and the discussion is complete.

But, let’s assume SQL databases do not scale now, I believe they can be scaled without changes to the application logic.

The solution can be found in Sharding; which means partitioning the data between servers according to access patterns.  I propose automatic sharding which will take the database requests, at the client end, and redirect those requests to the machine(s) with the required shard of data.

Automatic sharding should be completely possible:  Database constraints reveal the strongly connected data, but also reveal the natural break lines in that data.  Application access patterns (from profiling) can provide evidence of about what tables do not change often, and what data is ripe for replication.

For example, the automatic sharder should “see” that partition by user id is effective because the data dependencies between users is quite small.   Furthermore, the mutually dependent portion of the database will consist of lookup tables, and other rarely changed data; which can be replicated given the few times it changes.

The relational database was designed around the independence of rows.  This row independence is necessary for highly parallel operations, which is exactly what sharding needs.   If it is true that the database community has been “trying to solve a problem for twenty years and still haven’t managed to come up with an obvious solution”, then I am dismayed:  After sharding a database or two, it should be obvious how to automate the sharding.

My guess is that the human body generates vitamin D from the radiation.

http://www.scienceboard.net/community/perspectives.122.html

I may soon be determined to replace WordPress: It loves to add formatting to my html, making my posts unreadable. Until the day comes that I can get off my lazy butt to do something about it, I will just add a link:

Equal Temperament is a Poor Approximation

Finally, a simple mathematical presentation tool for the web: ASCIIMathML

Check out the awesomeness:

`"ERROR" = sum_("over all Just Notes") (( "note" ^("round"(ln( "note" )/ln( "spacing" ))) - "note") / "note" /( "max"^|"power"\|))^2`

Thanks again to WordPress to make using this Javascript library just a little bit harder.

One of my favorite blogs, Good Math, Bad Math by Mark Chu-Carroll, has a short post on how fundamental the pentatonic scale is.

I am mildly disappointed with how Chu-Carroll, and many others, think it is amazing that the human mind is able to distinguish the 2/3 note ratio. If you watch the video on Chu-Carroll’s blog: The guy dances on stage as if he is jumping on the keys of a giant piano. He sings the first two notes (with a ratio of 3/2) to the audience. When the guy jumps on the next key up the scale, the audience sings back the third note without his help. The audience knows this third note is another 3/2 up the scale. There is nothing magical about the 2/3 ratio. The guy could do the same with 1/2 note ratios (whole octaves) with the same success, and maybe even the 5/4 ratio with the same success. (Maybe the latter with less success, but only because the notes are getting closer together and are harder for the audience to sing).

I consider the 1/2 ratio “fundamental”, so fundamental that it is given the name “octave”, and all equal temperament scales have been made to align to the octave.

I solo jumped from a plane and survived!

Paul, the jump master, my sister Karina, and I are in a small plane about 3000 feet in the air. The door opened, Paul got in position 1/2 out of the plane. He says “Get ready!” I climb out on the struts and hang from my hands, with feet dangling in the wind. The wind is not too strong, maximum 100km/h, but strong enough to hold some of my weight and make holding the wing strut easier than when training on ground. I look at the jump master to indicate I am ready. He yells “Go!”, and I let myself drop.

The first second is pure panic because I was falling. I should have been looking up at the plane as I fell, but I was focused on the ground below. I was too busy counting.

Seconds 2 to 4, I could feel the parachute pulling harder and harder as it unfurls.

Second 5 and 6 I was still counting (as I was trained) but the chute had clearly deployed fully.

I tried to check my canopy, but I could not look up! I quickly figured out the lines were twisted right behind my head, (twisted lines are common). I reached back to pull them apart. I found the handles for steering above my head, and did my steering checks.

The rest was following commands from the radio operator so I landed in the right spot.

Here are all the things I did wrong (good thing it did not matter)

• I Probably did not arch my arms and head back enough in the first couple of seconds, and that is why the twist in the lines was right behind my head
• Never did check my altimeter. Once the radio worked I enjoyed the view too much.
• Never cheched the wind socks to know which way I should be going. Again, the radio guy did that for me.
• Did not put my feet straight below me. The ground looks like it is coming at you some reasonable horizontal speed, but that is an illusion. I landed on my feet, but then fell to my ass.

I mentioned my sister. Here she is, just about to land!

I have been “training” almost all month. I do fall of the wagon around the weekends, but at least I have not given up. But lately I have worried that I may not be trying my hardest. So, to ensure I am not subconsciously cheating during my “training” I got myself a heart rate monitor.

I found out yesterday that, not only am I out of shape, I am a wimp too:

1. While sitting in my programming chair, my resting heart rate is 97bpm, which puts me in the “light exercise” category. (out of shape)
2. Just getting up to walk puts my heart rate at 110bpm
3. While running hard, with my shins are killing me and gasping for breath, my average heart rate is only 165.

This final point shows how much of a wimp I am. My maximum heart rate (MHR) got to 195 (momentarily, and may be a reading error), which means I can only endure 69% MHR which is classified as “weight control”. Wikipedia says my maximum heart rate is 181, which would means I am training at 80%, which is classified as “Aerobic”. Beyond that the pain is unbearable.

Despite my disappointment, I am happy I have a real metric to compare against. Now I can set my target heart rate to just below my first day average: this way my watch will beep when I am slacking. More good news: I am so heavy that my 7 min jog burnt 170 calories!

In conclusion, there is little to distinguish me from a bowl of jelly. Tim may have to scoop me up when I can’t do those longer portages.

Introduction

I am going on the most fantastic Algonquin trip with my long time, and very good, friend Tim.
 

For those that do not know Tim, here is his IM image. As you can see, he is a healthy and fit individual. He takes care of himself, and he prefers walking over driving.

And here am I. Do not let my stunning good looks, and my slightly overweight body let you overestimate my heath: I prefer driving to the corner store, even though it would take less time to walk. I get grumpy when I have to get out of my programming chair, and I am winded after going up a flight of stairs. I am extremely out of shape. So, to prepare for this trip, I must exercise daily with the hope I am fit enough to not die of exhaustion 1/2 way into the park.

Aside the first couple of days, all days are reasonable so that we may catch-up to the plan should the first days prove too hard, or we have bad weather.

Sunday June 21st

We get to the Tim Lake campsite late in the afternoon. The first part of Tim river may have low water, and we will be asking about that. If it does, we will be entering via entry point 3 and camping on Butt Lake the first night.

Monday June 22nd

The first full day is traveling the Tim River all the way to Big Trout Lake.

I hope no hungry bears catch us. I hear the bears of Tim River crouch patiently in the bushes and pounce on unsuspecting paddlers. The number of bears have grown considerably in the last decade because paddlers are an excellent food supply for the bears.

Do not worry, I have a plan: I will just punch the aggressive bears in the nose. That’ll stop them.

Tuesday June 23rd

The trip to Cedar Lake will be the hardest and longest. Tim and I have a choice between going through Catfish Lake, or taking the Nipissing River. The latter has less total portage length and goes downriver, but the route is longer and our canoe is not designed for white-water, should we encounter it.

This leg of the trip is were I am most likely to die of exhaustion, or get eaten to death by blackflies. If I do die, tell my wife my body will be found off to the side of one of the longer portages.

Wednesday June 24

Now the trip gets easy. Tim and I get to travel the Petawawa River, and probably have camp setup by afternoon on Francis Lake. Time permitting we may go see Eustache Lake:

The deepest lake in Algonquin, Eustache is ringed by 25m cliffs, which continue down through its amazingly clear waters for another 90m

Thursday June 25th

This day will be upriver, and have a 2.2km portage. We should have time to check out the “huge Red Pine”, assuming they are still there, and the beetles didn’t kill them all.

Friday June 26th

The final day starts with a grueling 5.3km portage, and is one of the last places you can find my exhausted body should I not return. This may take all day to do, but luckily we will have eaten all our food so are packs should be light.

Hopefully we end up at my favorite little island in Opeongo. The same island my Dad brought my sister and I to on my first Algonquin trip, and the same island I bring almost everyone that comes to Algonquin with me.

Saturday June 27th

We paddle an hour down the South Arm, pack the car, and head home.

Finally, a name for what I am interested in defeating:  Page’s Law.  I advocate a proliferation of domain specific languages, and higher level languages to help the programmer become more productive.  Unfortunatly, these high-level laguages make less (compile-time) assumptions and run much slower.

I do not know what Google has planned, but my plan is to focus on optimizing the abstract high level set-operations, and optimize the common compiler and interpreter operations.  This is nothing new, just they have not been done in a single system, during runtime, before.