Kyle Lahnakoski

Just One More Layer of Indirection
(Trying to achieve stable orbit with sufficient architecture)

Archive for the ‘Rants’ Category

« Older Entries

96px != 1inch

Sunday, August 22nd, 2010

I find it ridiculous that Firefox will be redefining CSS Units. The plan is to redefine the physical CSS dimensions so that 1in = 96px, and add new physical dimension called “mozmm”, for the physical millimeter. Let me make this clear: Currently CSS defines “1mm” to mean one physical mm, Firefox whats to change “1mm” to mean 3.779px, and define “1mozmm” to mean one physical millimeter.

I hope it is not just me that asks “Why!?!?!?”.

This is how I see it:

  • If web developers are using physical units wrong, then let their pages render wrong
  • Even so, if you want to render badly designed pages, then just lie about the pixels/inch.

Posted in Economy, Personal, Rants, Technology | No Comments »

Are Ad Servers Bogging Down the Web?

Monday, November 30th, 2009

Slashdot brings up a point I complain about: Ad servers are slowing down the web.

I do not use web applications because they are slow. I do not know what people do to pass the time when they wait for each page to load. Using web mail, and adding an attachment makes you feel like you wasted precious time.

The web is mostly slow because of server latency. Especially “waiting for …” whatever ad server has been bogged down. I particularly dislike the sites that also use the slow Google Analytics servers.

Posted in Economy, Languages, Rants, Technology | No Comments »

Try an Index instead of Changing Your Infrastructure

Sunday, November 15th, 2009

One thing that disturbs me is the proliferation evil agents who love key-value stores. Especially those that love key-value stores in a latency-infested cloud. What upsets me more are the infinitely confused people who believe a database is *worse* than their key-value storage.

Here is one where Ian prefers Cassandra over proper database indexes:

Looking to the future with Cassandra

For some reason, Ian has compared his terrible query to his optimized Cassandra implementation. The query (and schema) are so bad, I suspect it’s a straw man.

Ian does not provide the SQL which makes him conclude that “Computing the intersection with a JOIN is much too slow in MySQL, so we have to do it in PHP.”. Any statement that implies a join is done faster outside the database should set of warning bells: The database should have all the information required to make your queries fast. If this is not the case, then something is seriously wrong with your indexes.

An all-database solution, even if it is a stored procedure, will be faster than a networked solution just because of latency. Personally, I have found returning a few hundred extra rows from a single “close enough” query significantly faster than issuing two queries with perfect results: Latency is your biggest enemy.

Let’s look at the Digg schema provided:

CREATE TABLE 'Diggs' (
  'id'      INT(11),
  'itemid'  INT(11),
  'userid'  INT(11),
  'digdate' DATETIME,
  PRIMARY KEY ('id'),
  KEY 'user'  ('userid'),
  KEY 'item'  ('itemid')
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 

CREATE TABLE 'Friends' (
  'id'           INT(10) AUTO_INCREMENT,
  'userid'       INT(10),
  'username'     VARCHAR(15),
  'friendid'     INT(10),
  'friendname'   VARCHAR(15),
  'mutual'       TINYINT(1),
  'date_created' DATETIME,
  PRIMARY KEY                ('id'),

  UNIQUE KEY 'Friend_unique' ('userid','friendid'),
  KEY        'Friend_friend' ('friendid')

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Some changes to the indexes would help:

  1. KEY ‘user’ (‘userid’) – does not help much when a user has digged many items: The index will help by pointing to all the actual ‘Diggs’ records, but the database will have to load every one of those blocks from disk to get that information (very likely one block per record). I would have suggested UNIQUE KEY ‘user’ (‘userid’, ‘itemid’, ‘digdate’) – This would have allowed the query to simply use the index, and not have to go back to the massive, unsorted, ‘Diggs’ table.
  2. UNIQUE KEY ‘Friend_unique’ (‘userid’,‘friendid’) – Seems to be the correct index to for “Query Friends for all my friends.”; this should be a single block lookup. There is no reason this should take 1.5seconds.
  3. KEY ‘Friend_friend’ (‘friendid’) – Maybe instead, Ian intended to have a list of all users that made ‘me’ a friend, rather than all users ‘I’ have befriended. This certainly explains the 1.5sec response time. In this case, the index should be expanded so the table blocks do not need to be loaded: UNIQUE KEY ‘Friend_friend’ (‘friendid’, ‘userid’).
  4. Maybe MySQL is poor database and loads the original records during a query even if the columns are not needed

Anyone who may be complaining about the extra disk space required to write these indexes should note that Ian’s Cassandra implementation consumes much more space than I am advocating here.

Even *IF* the Digg database is so big that the index lookups take too long, we should realize that we can pre-compute query results in the database, just like in Ian’s Cassandra implementation. If the database does not have materialized views, we can always add triggers to do the job ourselves. The former is still a limited technology, and the latter is quite messy, but both are better than changing your whole platform.

Finally, it seems Ian is trying to optimize for the worst case: “Kevin Rose, for example, has 40,000 followers”. I disagree with changing your infrastructure for a single use case for a minority of users, but that is a business decision that involves more issues than Ian’s blog entry can be expected to consider.

In conclusion, I am angry that the human race has lost another soul to the legion of key-value fanatics. I am further incensed that apparently 298 other nameless souls have followed Ian into the pits of hell. (298 diggs at time of writing).

Posted in Coding, Rants | No Comments »

RMS vs Miguel

Friday, November 13th, 2009

Introduction

Last month RMS and Miguel had a disagreement. Only now have I had the time to write out my thoughts.

RMS
Miguel

Miguel is Overly Optimistic

First, I can agree with Miguel when he says

“I know that there are great people working for the company,…”

but I take issue with the second half of his statement,

“…and I know many people inside Microsoft that are steering the company towards being a community citizen.”.

I have no doubt that Microsoft’s employees are trying to steer the company towards being a community citizen. But Miguel has an implicit trust that shareholders will not take back that steering wheel and drive in the opposite direction. That is where I oppose Miguel’s optimism.

Microsoft shareholders have been sitting on a goldmine for the last 20 years. Sure, Microsoft has been making reasonable products over the years, but it’s profitability is primarily due to the great waves of money in the world economy, generated by ever-increasing public and private debt. The population spent money they did not have for any nifty software feature. Microsoft developers benefit in this environment of free money because the shareholders find it’s easy to be altruistic when profits are high. I even contend that free software has had a hard time competing because money itself is (apparently) free.

The good times for Microsoft will not last. I believe the next decade will show how cruel the shareholder can be to “open source”. There are two main forces at work which will make the Microsoft shareholders act much more ruthless, and probably more shortsighted.

  1. A poorer user base: Money will be tight, either because of domestic inflation, or lack of liquidity. Microsoft faces an unrelenting barrage of competition from Free and Open Source software. People and corporations will have a greater incentive to use free software to reduce their spending.

  2. Profitable innovation is reaching it’s limit: The success stories of the last ten years depend on massive user bases: 10 million, 100 million or more. Each individual user only contributes pennies, if even, to overall revenue. Revenue per user is only going to go down further. I do not want to go into detail about why I believe this is true, but generally the software industry had matured: Software for the commoners has been built, and software niches are filled.

Microsoft is stuck between this innovation limit, and Free software’s relentless catch-up. Microsoft will feel the squeeze and start acting like most corporations that see their business model die: Sue.

I suspect that this fear of mine is just like Stallman’s, and I do not consider it irrational. Microsoft has every right to protect it’s patents. From the shareholder perspective, it must protect it’s patents when net-losses threaten the company.

Miguel gets Distracted

Miguel says:

“Working at CodePlex is a great way of helping steer Microsoft in the right direction. But to Richard, this simply does not compute.”

Miguel has fallen for the classic work-with-them-instead-of-against-them. Just like the environmentalist employed by a big oil corporation; he is told that he will help the company along the right path. But really, his employment/involvement is spin for advertisement, and for government tax rebates. The company would do the same without the environmentalist’s help, only now the environmentalists have one less advocate.

Microsoft will have done fine without Miguel. But now Microsoft can now advertise Miguel to the Open Source community, and hopefully Microsoft has distracted Miguel enough from being a competitive threat.

Miguel is motivated by Profit

Open Source, which I define as Open Source *not* including Free Software, is sold to the public as a compromise between the GPL and proprietary licensing. Really, Open Source is an advertising scheme used to acquire important tech-savvy users which install software on the majority of our machines. Open Source has the secondary goal of gaining some free debugging. Both goals include not giving back.

Open Source Profiteering is pragmatic, effective, and efficient at bringing products to market, but this is a short sighted goal. Open Source has it’s place, it is necessary, does some good, and it is what I would do if I ever released software people wanted. That does not mean I have to like it: I like steak, but I don’t like the thought of chopping up cows.

“Richard Stallman frequently conjures bogeymen to rally his base. Sometimes it is Microsoft, sometimes he makes up facts and sometimes he even attacks his own community”.

First, Stallman is sometimes wrong, after all he is only human. But to say he is conjuring bogeymen is misleading. Stallman is only issuing warnings of possible problems. He advocates actions that should be taken to avoid those possible problems.

Stallman is thinking long term, which is necessarily hard to be accurate. Stallman may misidentify the benign as threats (like with .Net, maybe), or he may identify threats as benign (I personally wanted something like GPL v3 back in the 90’s). Miguel does not attempt to see the long term, nor appreciate the difficulty in doing so. When Miguel hears warnings about .Net, and Microsoft, but Miguel “knows” there is no danger over the next year, he simply assumes Stallman is fear mongering.

Stallman is not motivated by profit, and Miguel does not understand this. Miguel assumes his goals are shared by all others. When Miguel says:

“Looking at opportunities where others see hopelessness. … I rather work on constructive solutions to problems than moan and complain.”

Miguel assumes the opportunities he finds, and the constructive solutions he invents would be lauded by any reasonable person. Miguel is wrong. Opportunities are defined by goals. Constructive solutions, any solutions really, are defined by goals. Miguel’s goals are profit. Miguel’s found opportunities and creative solutions are of no interest to Stallman.

Stallman is not a salesman. If Stallman was sent to Africa he would not see the shoeless as “hopeless” situation, nor as an “opportunity”, because both perspectives require a profit goal. Stallman would probably walk shoeless with the natives, and eat some good food, and maybe teach them to make their own shoes.

Conclusion

Miguel’s perspective is that of a short sighted, pragmatic, profiteer. As such, he makes a few wrong statements and conclusions:

  1. Microsoft’s employees can control the company direction – No, shareholders control the company direction.
  2. Stallman is pessimistic because he does not laud the “opportunities” and “constructive solutions” – No, Stallman simply does not share Miguel’s profit motivation, so those “opportunities” and “constructive solutions” are not.
  3. Stallman is fear mongering – No, Stallman’s simply warning others of possible threats to Free Software.

Posted in Rants | No Comments »

Protected: My Distain for Those Security Questions

Monday, May 18th, 2009

This post is password protected. To view it please enter your password below:


Posted in Rants, Uncategorized | Enter your password to view comments

Snow!!

Tuesday, April 7th, 2009

I just packed my snowblower away, the flowers are probably dead, and we must put snowsuits on the girls.

From the window upstairs…

I think I took out the lawn furniture too soon…

Beautiful though.

Posted in Rants | No Comments »

Death of LtU

Wednesday, March 25th, 2009

I will explain this pattern in a moment, but first I will say I have only seen this pattern a couple of times before. This pattern indicates imminent death for the newsgroup, mail list, or community website. I am posting this to test my prediction: LtU’s death is certain.

History

Highly technical newsgroups, mail lists, and web communities start off with a (small) excited family of founders interested in a specific domain. In the case of LtU, it is the domain of programming language theory. This family explores the defined domain, sharing technical articles, participating in interesting discussions, and combining the newly discovered ideas in ways new to them. Along the way they teach themselves and maybe some others (experts) along the way. But over time, the combined archives accumulate close to all the relevant papers, and all that needs to be said on that domain. The founders and experts start to have a difficult time finding ideas that simulate their imagination like they experienced in the past. Discussion amoung the founders and experts stagnate.

Meanwhile, the community collects many members. Most are interested in the topic, but not to the extent the founders and experts are. Some see the community an opportunity to learn more (newbies), and others see it as peripheral to what interests them more (trolls), and others just listen to the interesting things being posted (lurkers). In any case, this larger community will contribute questions and ideas that are mostly off-topic. So long as the discussion between the founders and experts produces enough new material, and the relative number of newbies and trolls is small, the community stays vibrant. Unfortunately, the growth of the off-topic community, combined with the increasing difficulty of the founders and experts to produce stimulating content eventually tips the balance from mostly on-topic to mostly off-topic.

The founders and experts are rightly dismayed at the large amount of “low quality” community discussion. This leads to the meta discussion…

http://lambda-the-ultimate.org/node/3219

…and then the collapse. The founders are unable to recognize the finiteness of their domain. Like a drug, they wait patiently for some new exciting concept to discuss. Meanwhile they prevent or discourage “off-topic” discussion. The lack of new content kills the community. The number of members drops off precipitously, and the community dies. The community leaves behind an archive and/or a wiki, with one or two diehard maintainers.

Problem Analysis

Community collapse is not inevitable. In my view there are three issues conspiring to destroy the community. Removing any one of these issues allows the community to stay alive.

  1. The domain is finite
  2. The number of experts is too small.
  3. The experts do not want to re-discuss issues

Expand the Domain – The finiteness of the discussion domain is the biggest reason there is a problem. The community/website must expand to include the prevalent off-topic themes. Hopefully the founders and experts have interest in these new topics, because I imagine the community will need to help to clearly define the new topics, and police them appropriately. The chances of a community expanding instead of collapsing is rare; the founders and experts need to play a larger role in moderation as the amount of discussion grows. Even if new moderators are recruited, founders and experts must realize there will be more discussion than they can digest.

I believe this was the direction took by the many (public) technical forums out there. I admit these may not have been born from academic-level community, but behind those sites is the discipline to support diverse and, admittedly, non-technical discussion.

Increase the Number of Experts – The community is in crisis because there are not enough on-topic posts. If the number of experts is increased, the number of posts should increase also. The problem with this solution is that there are not enough active experts. If there are, then they are usually found in academia publishing peer-reviewed papers, in which case they already have their own community: with journals and meetings, far superior to internet discussion.

Maintaining expertise of the members, and the quality of the discussion, has be done, and it takes the form of academic communities. The discussion is not intense because the reality is there is not many new things coming out of any one domain. For now, the best a community can do is distribute the papers of real experts. Maybe in time the major journals will start their own forums: Posting allowed only by invitation, of course.

Keep experts excited – Any one person will eventually loose interest in a fixed domain, therefore a consistent stream of new experts is required to replace the old. This means the founders and old-guard experts must stand down and pass the discussion on to a new generation of experts. This will inevitably result in many re-discussions of old topics, but the community will stay alive discussing past topics while waiting for the occasional new idea to arrive. As with all re-birth, do not be surprised if the offspring does not resemble the parent. How many founders would allow that to happen to their pet project?

I have seen successful handoffs before; the current experts do not have the time to maintain the list. I have seen the discussion change as the priorities of the new experts dominated the discussion. Problem is that the next generation of experts can not release control; and the community collapses eventually anyway.

Conclusion

I will put a line in the sand, if only to test my abilities of prognostication: LtU does not have the will or resources to expand it’s website to include a more diverse set if discussions. LtU can not try to compete with academia’s clearly superior quality discussion. LtU moderators can not stomach releasing control the hoards of off-topic trolls.

The LtU community is about to die.

Posted in Rants, Updates | No Comments »

Simple Lack of Oversite

Thursday, February 26th, 2009

The Clinton administration altered the Community Reinvestment Act in 1995. Despite what the GOP operatives would have you believe, it was not the cause of the housing meltdown.

The relaxing of lending regulations did not cause this problem because the relaxed regulations only affected less than 20% of the banks making loans.

If you want to blame relaxed mortgage regulations, you should blame the complete lack of regulation in the mortgage industry. But, even this lack of regulation is not the problem. If a corporation thinks giving money to the poor for overpriced houses makes good business sense, then they should be allowed to do so. The real problem was that the creditors and shareholders were not overseeing these companies appropriately.

What I find abhorrent is the use of public money to bailout these same lazy creditors and shareholders.

** If your managed investments, like 401K or mutual funds, lost money then maybe it was because they chose non-voting shares, or they voted for the same-old-board, or they did not vote at all. I suggest you start a class-action lawsuit for their lack of due diligence: What did you pay those management fees for?

Posted in Rants, Technology, Updates | No Comments »

Digg

Tuesday, February 10th, 2009

I visited Digg today. I do not usually do so because it seems to select the most uninteresting of stories; I feel I am dumber for reading it.

Anyway, has Digg become a portal for the corporate media? Maybe I should save myself a click and just bookmark Fox News instead.

Posted in Rants | No Comments »

Wisdom

Saturday, February 7th, 2009

Shorter Paul Graham: “You can not have a useful conversation with people who take it personally.”

I will never get those minutes back.

Posted in Rants | No Comments »

« Older Entries
kyle@arcavia.com