Everything is a Freaking DNS problem

The age of #monitoringsucks is over, we're now transitioning into a #monitoringlove period.

That however doesn't mean al the work is done, we still need to do a lot of work and a lot of people are working on a lot of stuff.

Therefore like last year we are opening up our offices again right after Fosdem for a #monitoringlove hackfest

That's right on february 4 and 5 a bunch of people interrested to fix the problem will be meeting , discussing and hacking stuff together in Antwerp. In short a #monitoringlove hackathon

Inuits is opening up their offices for everybody who wants to join the effort Please let us (@KrisBuytaert) know if you want to join us in Antwerp. We'll provide caffeine, wireless, chairs and some snacks.

Please register upfront at : http://monitoringlove2013.eventbrite.com/

Obviously if you can't make it to Antwerp you can join the effort on ##monitoringsucks on Freenode or on Twitter.

The location will be Duboistraat 50 , Antwerp
It is about 10 minutes walk from the Antwerp Central Trainstation
Depending on Traffic Antwerp is about half an hour north of Brussels and there are hotels at walking distance from the venue.

Plenty of parking space is available on the other side of the Park

Read last years report http://www.krisbuytaert.be/blog/we-didnt-fix-it to get an idea of what will happen...

PS. Yes I`m trying to get another event of the ground the days before Fosdem but I`m still awaiting confirmation of the venue ..

With 2 of the bigger Open Source projects I care about talking about certifications programs questions pop up again ...

Should we certify ourselves ?

So let me tell you about my experiences in getting Open Source related Certifications ..

Over a decade ago, (2001) when RedHat was still Redhat and not yet Fedora the company I was working for was about to partner with RedHat and needed to get a number of people certified for that.

So I took the challenge, I bored myselve to death during a 4 day RedHat fast track training and set out to do the exam the next day. Obviosly I scored pretty well given my yearlong experience in the subject. Back then I was told that I scored the one but European Record on the exam which was actually held by another collegue (hey Ico) , our CTO however was not amused when I told that I could have scored better but I didn't bother running a chkconfig smb  on since I didn't see the use in using windows fileshares in a unix environment (Yes I was young , we're all allowd to make stupid mistakes :))

So I was certified, we were expecting the requests to flow in en masse ... nothing happened... not a single customer request... If I recall correctly we got 2 requests for certified engineers over the course of the following years. One was from a customer that wanted to have us do some junior level sysadmin work on their systems which we didn't care about, we proposed a more junior profile, but they insisted on having someone who was certified, The other one was from a Large institution that wanted certified people for their RedHat support, only to quickly learn that the budget they had planned for this project was about half the rate we usually charged ..

When RedHat introduced their certified Architect program my answer was, sure .. if you bring us the customer that will make the investment worthwhile , guess what..

My second experience with Open Source certification came a couple of years later with MySQL, same story partnering etc, . only this time our trainer had put some focus on a couple of slides during the training (Hi Tobias) and during the exam indeed one of those questions popped up, The correct answer to "What are the core values of MySQL AB" was "We reply to email" , I stood up and left the exam ...
I ranted about this to a number of people including Roland Bouman who back then was just starting on the MySQL (NDBD) Cluster certifciation track and I assisted him in making the book to study for that exam better.
Once again .. pretty much no one asked for MySQL certification in Europe back in those days (2007 ?)

I won't go deeper into discussing the Xen certification I got from Citrix, but it involved correcting slides from the presenters at the first European training.

Based on my experience with these certifications in Belgium/Europe you can see that I`m not a big fan of certifications I have not seen a reason for me to certify yet

I actually think that noone within the Open Source community should be looking for certification, we should be looking for people that are active in the community and that are contributing to projects.
Unlike in the proprietary world where you have to cough up tons of money in order to get a license to play with a tool and learn itl In the open source world with projects such as both Drupal and Puppet, there are absolutely no excuses for Junior people not to engage and prove themselves. they have full access to anything they need, the only thing they need to do is want to get involved.

Sadly this world however is still full of incompetent recruiters, middlemarket agencies that will never understand this and will ask for cerftifications of some kind. My fear is indeed that there will be a group of mediocre but certified developers swarming these growing markets at dumping rates and that the people with the real experience that have been involved in the communities for ages already will be the ones pulling the short straw.

Anyhow ... in just a short couple of years everything will be fine again .. as by then my RHCE will be current again and the incompetent recruiters that need people that are RedHat 7 certified will start calling me by the dozen.

I`m parsing the responses of the Deploying Drupal survey I started a couple of months ago (more on that later)

One of the questions in the survey is "What is devops" , apparently when you ask a zillion people (ok ok, just a large bunch of Tweeps..), you get a large amount of different answers ranging from totally wrong to spot on.

So let's go over them and see what we can learn from them ..

The most Wrong definition one can give is probably :

• A buzzword

I think we've long passed the buzzword phase, definitely since it's not new, it's a new term we put to an existing practice. A new term that gives a lot of people that were already doing devops , a common word to dicuss about it. Also lots of people still seem to think that devops is a specific role, a job description , that it points to a specific group of people doing a certain job, it's not . Yes you'll see a lot of organisations looing for devops people, and giving them a devops job title. But it's kinda hard to be the only one doing devops in an organisation.

I described one of my current roles as Devops Kickstarter, it pretty much describes what I`m doing and it does contain devops :)

But devops also isn't

• The connection between operations and development.
• people that keep it running
• crazy little fellows who find beauty in black/white letters( aka code) rather than a view like that of Taj in a full moon light.
• the combination of developer and operations into one overall functionality
• The perfect mixture between a developer and a system engineer. Someone who can optimize and simplify certain flows that are required by developers and system engineers, but sometimes are just outside of the scope for both of them.
• Proxy between developer and management
• The people in charge of the build/release cycle and planning.
• A creature, made from 8-bit cells, with the knowledge of a seasoned developer, the skillset of a trained systems engineer and the perseverence of a true hacker.
• The people filling the gap between the developer world and the sysadmin world. They understand dev. issues and system issues as well. They use tools from both world to solve them.

Or

• Developers looking at the operations of the company and how we can save the company time and money

And it's definitely not

• Someone who mixes both a sysop and dev duties
• developers who know how to deploy and manage sites, including content and configuration.
• I believe there's a thin line line between Ops and Devs where we need to do parts of each others jobs (or at least try) to reach our common goal..
• A developer that creates and maintains environments tools to help other developers be more successful in building and releasing new products
• Developers who also do IT operations, or visa versa.
• Software developers that support development teams and assist with infrastructure systems

So no, developers that take on systems roles next to their own role and want to go for NoOps isn't feasable at all ..you really want collaboration, you want people with different skillsets that (try to) understand eachoter and (try to) work together towards a common goal.

Devops is also not just infrastructure as code

• Writing software to manage operations
• system administrators with a development culture.
• Bring code management to operations, automating system admin tasks.
• The melding of the art of Systems Administration and the skill of development with a focus on automation. A side effect of devops is the tearing down of the virtual wall that has existed between SA's and developers.
• Infrastructure as code.
• Applying some of the development worlds techniques (eg source control, builds, testing etc) to the operations world.
• Code for infrastructure

Sure infastructure as code is a big part of the Automation part listed in CAMS, but just because you are doing puppet/chef doesn't mean you are doing devops.
Devops is also not just continous delivery

• A way to let operations deploy sites in regular intervals to enable developers to interact on the systems earlier and make deployments easier.
• Devops is the process of how you go from development to release.

Obviously lots of people doing devops also often try to achieve Continuous delivery, but just like Infrastructure as Code it devops is not limited to that :)

But I guess the truth is somewhere in the definitions below ...

• That sweet spot between "operating system" or platform stack and the application layer. It is wanting sys admins who are willing to go beyond the normal package installers, and developers who know how to make their platform hum with their application.
• Breaking the wall between dev and ops in the same way agile breaks the wall between business and dev e.g. coming to terms with changing requirements, iterative cycles
• Not being an arsehole!
• Sysadmin best-practise, using configuration as code, and facilitating communication between sysadmins and developers, with each understanding and participating in the activities of the other.
• Devops is both the process of developers and system operators working closer together, as well as people who know (or who have worked in) both development and system operations.
• Culture collaboration, tool-chains
• Removing barriers to communication and efficiency through shared vocabulary, ideals, and business objectives to to deliver value.
• A set of principles and good practices to improve the interactions between Operations and Development.
• Collaboration between developers and sysadmins to work towards more reliable platforms
• Building a bridge between development and operations
• The systematic process of building, deploying, managing, and using an application or group of applications such as a drupal site.
• Devops is collaboration and Integration between Software Development and System Administration.
• Devops is an emerging set of principles, methods and practices for communication, collaboration and integration between software development (application/software engineering) and IT operations (systems administration/infrastructure) professionals.[1] It has developed in response to the emerging understanding of the interdependence and importance of both the development and operations disciplines in meeting an organization's goal of rapidly producing software products and services.
• bringing together technology (development) & content (management) closer together
• Making developers and admins understand each other.
• Communication between developers and systems folk.
• a cultural movement to improve agility between dev and ops
• The cultural extension of agile to bring operations into development teams.
• Tight collaboration of developers, operations team (sys admins) and QA-team.

But I can only conclude that there is a huge amount of evangelisation that still needs to be done, Lots of people still don't understand what devops is , or have a totally different view on it.

A number of technology conferences are and have taken up devops as a part of their conference program, inviting experienced people from outside of their focus field to talk about how they improve the quality of life !

There is still a large number of devops related problems to solve, so that's what I`ll be doing in 2012

Patrick posted his upcoming conference schedule for the next couple of months.
as you can see there are a comple of overlapping conferences :)

Conferences I'm speaking at or likely to attend are:

• The first week of October I`ll be in the Valley , I`ll be late for Jenkinsconf but I hope to pick up some events while I`m there.. suggestions are welcome , I`m also heading back to Europe earlier than planned so I will miss BadCamp :( ...
• Devopsdays Goteborg, Sweden : October 14,15 - The yearly Europe devops event is happening in Goteborg this time. It's going to be really exciting this time , as the theme is inclusive. Eploring the boundaries of devops, I`m once again in the organization of this conference.
• T-Dose 2011, The Technical Dutch Open Source Event, on 5 and 6 november 2011 , I will be talking again about my experiences with complex Puppet setups
• Citconf , London: November 11-12 - All you ever wanted to know about Continuous Integration. Period, registered, haven't booked flights yet.
• Cloudcamp Belgium: November 21 - I'm looking forward to this year's event, as there will likely more practioners and less marketing folks.
• Lisa 2011, Boston, US, I`m giving an Invited talk titled , Devops: The past and futre are here, It's just not evenly distributed (yet), and I`ll be on a panel titled What Will Be HOt Next Year, really looking forward to this one :)
• Fosdem.org will take place on 4 and 5 February 2012 , and as every year since it inception I'll be there
• The UKUUG rebranded to FlossUK , they are hosting their Annual Spring conference from 20th to 22nd March in Edinburgh , given their refound focus it will be even more interresting !
• And as announced earlier this week Loadays.org will take place in Antwerp again this year on 31/3/2012 and 1/4/2012 , as the previous years I`m co organizing this conference

And yes, I do work from time to time. Just that these conferences are a great way to capture and share new ideas. All worth it!

Lenz gave the good example so I`ll follow :)

Next weekend saturday I`ll be giving a talk about devops at StartUp Weekend Brussels, from what I've read so far it promises to be an audience that needs the talk,

The week after I`ll be speaking at the DrupalDevDays, again about devops , however this time with a touch of Drupal , giving a devops talk at Devoxx last year to a Java audience learned me that the devops evangelist need to go outside of their usual conference audiences and als talk to the people that are usually in the other silos.

Next march I`ll be speaking at the UKUUG spring conference in Leeds this time about my experiences on High Availability with Pacemaker

And who knows I might squeeze in a talk at Load this year also ..

If you are around at one of these confs and you want to talk Devops, Clustering, sipx or just have a beer .. don't hesitate ! There's already plenty of people promising me beers , and some even sushi :)

That's the question Xavier asks in his blog entry titled
Security: DIY or Plug’n'Play

To me the answer is simple, most of the appliances I ran into so far have no way of configuring them apart from the ugly webgui they ship with their device. That means that I can't integrate them with the configuration management framework I have in place for the rest of the infrastructure. There is no way to automatically modify e.g firewall rules together with the relocation of a service which does happen automatically, and there is always some kind of manual interaction required. Applicances tend to sit on a island, either stay un managed ( be honest when's the last time you upgraded the firmware of that terminal server ? ) , or take a lot of additional efort to manage manually. They require yet another set of tools than the set you are already using to manage your network.
They don't integrate with your backup strategy, and don't tell me they all come with perfect MIB's.

There's other arguments one could bring up against appliances, obviously people can spread fud about some organisation alledgedly paying people to put backdoors in certain operation systems.. so why would they not pay people to put backdoors in appliances , they don't even need to hide them in there .. but my main concern is manageability .. and only a web gui to manage the box to me just means that the vendor hates me and dooesn't want my business

A good Appliance (either security or other type) needs to provide me an API that I can use to configure it, in all other cases I prefer a DIY platform, as I can keep it in line with all my other tools, config mgmtn, deployment, upgrade strategies etc.

Mabye a last question for Xavier to finish my reply ... I`m wondering how Xavier thinks he kan achieve High-availability by using a Virtual environment for Virtual Appliances that are not cluster aware using the virtual environment. A fake comfortable feeling of higher availability , maybe.. but High Availability that I'd like to see.

A couple of weeks ago I ranted about being able to crash about any available music player on Fedora, and the gazillion bugs I filed for that ..

At last the the problem is solved, as mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=657971#c17
this is most probably a bug in ffmpeg which is provided by rpmfusion. and which is solved by installing gstreamer-plugins-ugly

At least it works for me :)

Apparently December is the month where everybody starts writing guest posts for other blogs.

Earlier this month I wrote an article with the title of this blog for Sysadvent ,

It's a sysadmin relative of the Perl Advent Calendar: One article for each day of December, ending on the 25th article. With the goals of of sharing, openness, and mentoring, we aim to provide great articles about systems administration topics written by fellow sysadmins

My article is here, but there's plenty more other articles written about a variety of topics, such as chef, tcpdump , how ls works, cucumber and Devops.

On the other side, Matthias over at Agile Web Development and Operations is hosting a series on Devops where lots of Devops Advocates and Evangelists are having their say about Devops ...

My entry about the Challenges the Devops Crowd faces was put online yesterday

You might have noticed that this blog stopped accepting comments about a month ago.. well. stopped accepting is a big word.. I was still accepting comments, only they were never submitted to the database and after entering a comment to my blog people ended up on a white page.

So upon returning from holliday I set out to debug the issue together with one of our Inuits Drupal geeks and quickly ran into the following error.

PHP Fatal error:  Call to a member function has_more_records() on a non-object in /somepath/modules/views/plugins/views_plugin_display.inc on line 1992, referer: http://www.krisbuytaert.be/blog/comment/reply/1014

So apparently my veasion of views 6.x-3.0-alpha3 didn't really like to play with Mollom,
I downgraded views again to 6.x-2.11 and Mollom started showing its Captcha's etc again .

So apart from wondering how I ended up installing that alpha3 version (I`m sure Drush didn't do that), all is back to normal. and you should be able to comment on this blog again

Following up on Wim's example