Kris Buytaert's blog

Oct 19 2009

Nines , Damn Nines and More Nines

Funny how different experiences lead to different evaluations of tools. The MySQL HA solutions the MySQL Performanceblog list, are almost listed in the complete opposited order of what my impressions are.

Ok agreed, I should probably not put my MySQL NDB experiences from 2-3 years ago with multiple Query of deaths and more problems than you into account anymore , but back then went in the list Less stable than a single node. I've had NDB POC setups going down for much more than 05:16 minutes
Ndb comes with a lot of restrictions, there are

As for MySQL on DRBD, I've said this before , I love DRBD, but having to wait for a long InnoDB recovery after a failover just kills your uptime ,
I remember being called by a customer during Fred last holiday who was waiting over 20 minutes for recovery , twice, so putting the DRBD/San setup second would not be my preference. But agreed .. it's only listed at 99.9% meaning almost 9 hours of downtime per year are allowed.

On the other hand we've seen database uptime of MySQL MultiMaster setups with Heartbeat reaching better figures than 99.99% Heck I've seen single nodes achieve better than 99.99% :)

So what does this teach us ... there is no golden rule for HA, lots of situations are different, it's the preferences of the customer, the size of the database, the kind of application , and much
more .. you always need to think and evaluate the environment ...

Oct 16 2009

Heartbeat 2 OpenAIS

While upgrading a pretty recent Heartbeat cluster to OpenAis earlier today I ran into the following weird situation

  1. Last updated: Fri Oct 16 08:50:03 2009
  2. Stack: openais
  3. Current DC: CO_NMS-1 - partition with quorum
  4. Version: 1.0.5-462f1569a43740667daf7b0f6b521742e9eb8fa7
  5. 4 Nodes configured, 2 expected votes
  6. 1 Resources configured.
  7. ============
  8.  
  9. Online: [ CO_NMS-1 CO_NMS-2 ]
  10. OFFLINE: [ co_nms-1 co_nms-2 ]

or

  1. crm(live)node# show
  2. co_nms-1(5c48ab4f-767f-e2dc-20ec-5969cddad152): normal
  3. co_nms-2(922ff786-eca9-bed0-d79d-8222727a2c5b): normal
  4. CO_NMS-1: normal
  5. CO_NMS-2: normal

Whohoo.. OpenAIS must have realized I have upperase and lowercase cores :)

Funny to see .. but quickly solved..

Oct 13 2009

Open Source, Open Core, Open ScoreCards

There is this constant discussion about Open Core vs Open Source vs Proprietary Software , Fauxpen Source, Open Source Business models etc.. you probably know all the usual suspects involved, first up lets agree that nobody will ever agree on what's best, (off course it's pure open source.. ) , but one of the important aspects is to know what values are important for you and your customers

Simon Phipps thinks we should build a scorecard that lists the different values we attach to a certain level of openness

He'd like a rating to questions such as , Is the license OSI-approved? , Is the copyright under diverse control? ,Is the community governance open?, Are external interfaces and formats standards compliant? , Does your community operate under a patent peace arrangement? Are trademarks community controlled? etc ..

Do we need one ? Matt Aslett , whom I finally met last week in London , thinks not as it will has the knife will cut on both sides, but then also thinks yes as it might clear up confusion to outsiders.

The comments on his post however is where the real discussion starts, the one where the Open Core fanboy tells about HIS customers not caring, and the Open Source zealots comment that the open core customers don't care as they already settle for the best of worst world (ok ok , I added that myselve :))

We are old and smart enough to decide ourselve .. aren't we ? Fact is that plenty of us already use this kind of scorecards for themselves, We prefer Open Source over Open Core , but still Open Core over proprietary software, we look at the community, we look at the source , we look at who's contributing and who's using. Sometimes we value a vibrant user community over a vibrant contributing community , sometimes we don't like projects with only contributions from 1 company .. we do that exercize daily

On the other hand, as Matthew Aslett states, the outsiders probably don't know yet, and as someone else in the discussion said, some customers are just stupid ..

Different Open Core vendors have different approaches, we should use our own brains to see the difference between Marketing Driven company squashing out buggy but open code and community driven company looking for a business model. If you, such as Sander claims constantly are trying to outsmart your community , don't you think your customers will realize that ?

An aftertought If Simon Phipps gets his wish granted, what do we do with the blogs etc, shouldn't Matt then change the subtitle of his blog to The business and politics of open core ? and his title to Open Core Executive to ? Or do we just call him the Richard Stallman of Open Core ?

Oct 13 2009

Everything is a Swedish dot Problem

Via @frank_be

.se goes down after a dns config issue
We have spoken to a number of industry insiders and what happened is that when updating the data, the script did not add a terminating “.” to the DNS records in the .se zone. That trailing dot is necessary in the settings for DNS to understand that “.se” is the top-level domain. It is a seemingly small detail, but without it, the whole DNS lookup chain broke down.

Oct 12 2009

Enterprise Open Source Adoption at BT, London

Repost from our corporate blog

Last monday some Inuits quickly crossed the channel for a day of speeches and talks regarding Open Source and its Adoption, the event organised at BT brought together a mixture of techies, legal persons and management to listen to and discuss about the current state of Enterprise Open Source adoption

The short introduction was done by JP of Confused In Calcutta , who mainly introduced Mark "I`m from outer space" Shuttleworth. Mark keynoted about Ubuntu .. he talked about Aubergine being the new Brown ... ranted (as everybody) about the Cloud , talked about a stronger focus to services rather than product building , talked about the ecosystem of "people close to you" for supporting solutions .

Steve Bouch, of the Synapse Project at BT discussed ao the decisions they had to make, the varialbes they took in to account when starting
to use Open Source , discussions such as the reputation of the project, the internal skill zet, the ease of importing and exporting data in and
out of the project, and the different license terms were covered.

Andrew Katz , legal guy , who's also working on the International Free and Open Source Software Law Review gave an overview of the legal impact of Opensource . We learned that his customers have similar questions as ours, such as when to publish the code, what code to publish, when to look for alternative licenses etc

There was avid discussion regarding the Affero GPL license and using Open Source as a key component to build your killar webinfrastructure while not contributing back etc.

During the pannel discussion questions like Software Patents, and the fact that Ubuntu One won't be fully Open Source were tackled For us , most of the event was confirmation that what we are doing is the right thing to do, And that is worth a lot !

Oct 11 2009

What Kind of Open Source Fanboy are you ?

James Dixon has some polls on his blog, they let you choose between being a free software junkie, an open source hacker or what I'd call a an incompetent moron ;)

I had mostly 2's , no 3 in case you'd wonder.

Oct 11 2009

Monitoring MySQL

Ronald Bradford wants to know what kind of Monitoring you use..
He specifically wants to know about Alerting tools

There's different cases , looking at it from a full infrastructure point my current favourite is Zabbix or good old Nagios,

But when looking at it from a debugging perspective you have MySQLAR or Hyperic, but those aren't in the alerting list.

However, when you are building HA clusters, you have custom scripts running either from mon or from pacemaker ..

Still .. Ronald probably wants more input :)

Oct 09 2009

Why learn to type ?

When your machine knows what you mean ..

  1. [s3p-root@XMS-1 tomcat6]# crm configure
  2. crm(live)configure# bye
  3. [s3p-root@XMS-1 tomcat6]# crm confiure
  4. crm(live)configure# bye
  5. [s3p-root@XMS-1 tomcat6]# crm confiture
  6. crm(live)configure# bye
  7. [s3p-root@XMS-1 tomcat6]#

I'd better

  1. apt-get install coffee

Oct 08 2009

T-Dose 2009

As every year in october a small crowd navigated to Eindhoven to spend 2 days discussing Open Source , Free Software and Beer

The good part about T-Dose is the fact that it is so small, you actually get the chance to talk to a lot of people, you get the chance to sit down with them and play with the code they have been working on ,

T-Dose had talks on IPv6, Bind10 , Linux Auto Update USB, eTokens on Linux , VirtSec, Django , Puppet and other topics.
I actually missed talks that I had wanted to see.. just because I was in the middel of interresting discussions with people. That's totally different from some other conferences where I don't even make it to the talks because of the smalltalk :)

T-Dose social event is always nice .. and so is the post social social event. (you know .. where the beer comes into play...) Luckily the dutch guys know where to go .. places with Duvel, Chouffe and Kwak ...

On Sunday I did a rerun of my VirtSec talk , the discussion was different this time more open :)

After my talk Sejo gave his first talk on Djagios .. it went great .. albeit some members of the audience fiddling with his live demo.

(Ok I plead guilty.. but it was just too tempting..)

Oct 01 2009

It's the solutions you build with it !

I`m gonna have to quote Tarus once again :


For years now I’ve been struggling to educate the market on the fact that the business around open source software is not about software. It’s about solutions.

Let me repeat that .. it's not about selling software ...

It's about solutions

And obviously FLOSS is the Ideal platform to build value for your customers