abangratz - blag

Stuff that matters - at least for me.

Critique of Usersnap’s “8 Bad Habits”

Today, I read an article by Gregor Dorfbauer of Usersnap called 8 GOOD HABITS IN WEB DEVELOPMENT.

Because what I have to say on the matter would make a bad comment (tl;dr coming up), I decided to post it on my blog and refer to it.

While there are quite a few good ideas, it does seem that we have different goals. Let me address my ideas point by point.

1. Use a Bug Tracker

I agree wholeheartedly. But I don’t agree with the workflow presented. For most working teams I’ve been in, there’s no “person in charge of fixing ‘it’“. Whoever is next in line to pick up a task, picks it up. If the task is “fixing a bug”, said person fixes the bug. I also disagree with the “user story” format, for most kinds of bugs the “action, expectation, outcome” format is the best.

Also, humour is a good thing, but not in the commit messages. Same with insults etc. A nice summary is found at Tim Pope’s Note About Git Commit Messages. If humour makes the message easier to understand, by all means, one should use it. But it usually obscures things.

2. Take responsibility

That works if features/user stories are small enough and can be handled in one or two days by a developer (perferrably pair). And if they are able to break down user stories (features) in small enough tasks that are most likely finished at the end of the day. If they are working with a ticket system, there’s no reason to push work on others, but have everyone pick their tasks. Same goes for a Scrum/Kanban board. Pull vs push helps a ton! (For a definition of “finished”, please check the point “Testing”.)

Another aspect is that the developers should be involved in designing features. And they should make themselves heard if they think that the feature they just heard about is, let’s say, questionable. A different view and a following discussion brings insight and helps do develop understanding of the whole product and processes.

3. Fix and test

Yes and yes. But there’s depth missing for me. First: create Unit Tests. Make them small, fast, concise and descriptive. Second: write stack/functional tests that make sure that the unwanted behaviour is not reintroduced. Third: make it part of the acceptance test suite/protocol. Only if all three modes of tests agree that the feature is working as intended, then a feature or bug is “finished”. For a task, Unit Tests may suffice.

4. Plan

Yes. But don’t forget to involve developers early on, else everyone planning might be unpleasantly surprised by estimates and actual costs (time) for a feature.

5. Single Sign Off

That paragraph rubs me the wrong way. It smacks of a position that was called “Delivery Manager” by a company I won’t name here, and he was responsible for implementing Scrum, internal process management and release planning and improving productivity. It ended badly.

For web development, I would rather suggest something like Scott Chacon writes about in Github Flow. (Don’t confuse it with git-flow, which is trying to solve a social problem with a technical solution.) In web development, one wants to deploy continuously. Especially if work is done by a small team, finding someone who can “sign off” a release will suck, if said person is on sick leave, vacation, congresses, heck, if that person is on lunch break.

In web development, it is smart to implement the following two things: Continuous Integration and Continuous Deployment. And implementing something similar to (or exactly like) the Github pull requests, and rule that the authors of the pull request should not merge and close it, but others in the team should. With people handling those tasks responsibly, it will result in a regular and efficient code review. And if tests are missing, the reviewers must not merge the request. But as soon as the request is merged into master, all tests have to be run. On success, the current master branch must be deployed immediately.

This will mean that occasionally a useless feature will be shipped. But everyone will just learn from that experience. If the team is using enough tests, and is fast on reacting to user complaints, no (lasting) harm will be done.

And in the end, if the team is using Scrum, there is one Person responsible: the product owner. But then he’s only responsible for accepting or rejecting a feature. Not for the rest of the process.

If there are reasons that keep the team from doing real continous deployment, using a staging system and a process where the testers can release feature by feature is helpful. Most of the time it is better to live with a short lived feature that users are not happy with – In contrast to deploying huge releases with a large scope every once in a while.

6. Create Feature Teams

Yes and yes. Nothing to add.

7. Did We Mention Testing?

Yes, I guess the article did and I did. Let’s leave it at that. The topic is huge.

6. Always Keep Optimizing

I don’t like the word and it’s connotation. (Suggested reading: Premature Optimization). But then the article goes into more detail, and becomes a bit clearer. Still, I have issues with the way it is worded. Let’s analyze:

1. Make it work.

Yes. But what is “it”? The best way to do it is: Make sure that the test coverage is alright, make sure that the code works as intended. A good way is to use Test Driven Development: Make it red (write a test for a future feature), make it green (implement it so the test passes.)

2. Make it right / beautiful.

No, no, no. Make it right the first time. Implement it to specs (tests), before going to the next step: Refactoring. Refactoring is not making it beautiful. Refactoring is taking away until nothing else can be taken away lest it breaks. Avoid repetitions, redundancy, or violating the SOLID principles). Also a good idea is to practice the ‘boy scout practice’ of Clean Code fame: When touching a piece of code, leave it neater than you found it.

3. Make it fast.

No. Just don’t. If there are no problems, it’s fast enough. If you suspect or have problems: First step is measuring how fast a newly implemented feature’s parts really are.Next step is measuring how fast the whole stack is. If there are real performance issues, then it’s time to identify them. Stick to the paradigm: Make the common case fastest. Check for alternatives in flow and algorithms before trying something like re-writing it in C. But please, for the love of George (or the FSM), don’t just optimize for optimization’s sake.

Here ends the article.

All in all, an interesting article, but for me it just doesn’t hit the spot. Good basic ideas, though, and it got me writing again :)

Minimal Rails App for Gem Development

Surprisingly, the resources for testing gems that have a dependency to rails (or parts thereof) are few and far between. That’s why I decided to have a short howto here for you to peruse, in case you run into the same troubles as we did.

Given the fact that you would need a Rails stack to simulate dependencies, especially if you want to integrate tests, you are stuck with adding a Rails app.

Yes, you are reading correctly: in the end, you need a rails app. But fear not, it’s easier than it sounds.

First, you want to add the dependencies to the Gem spec file:

myproject.gemspec
1
2
3
4
5
6
Gem::Specification.new do |s|
  s.add_development_dependency(%q<rake>)
  s.add_development_dependency(%q<rspec>)
  s.add_development_dependency(%q<rails>)
  s.add_development_dependency(%q<rspec-rails>)
end

If you want to use rspec, rake and all the stuff for running your gems, add the needed Gems to the Gemfile:

Gemfile
1
2
3
4
5
6
group :development do
  gem 'rake'
  gem 'rspec'
  gem 'rails'
  gem 'rspec-rails'
end

Yes, we pull in the whole stack. You can experiment with adding only parts, but that might be hard to accomplish.

Next, you want to create the mini Rails app. The best place is your spec_helper.rb file (in case you are using rspec):

spec_helper.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
require 'action_controller/railtie'
require 'rspec/rails'

require 'myproject'

app = Class.new(Rails::Application)
app.config.secret_token = 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'
app.config.session_store :cookie_store, :key => '_myproject_session'
app.config.active_support.deprecation = :log
app.config.eager_load = false
app.config.root = File.dirname(__FILE__)
Rails.backtrace_cleaner.remove_silencers!
app.initialize!

This can be also used in test_helper.rb for minitest. And yes, that’s all.

If it doesn’t work for you, get back to me, please, I’m happy to help.

The source is mainly taken from our gem query-interface-server, and has been partially adopted from devise and need_label.

Git EEP!

Over the course of working with git, everyone encounters errors. Now, there are quite a few tutorials that explain stuff nicely - but git is very powerful, so even the easiest explanations are hidden beyond pages and pages of documentation.

I want to point out the most common error messages and resolve strategies.

Common to all of the following is: DON’T PANIC!

Read the error message and you will find a strategy for everything.

Merge Errors

Merge errors happen after a git merge, git rebase or git pull. The most common are

  1. Merge conflicts
  2. Avoiding overwriting of local changes

Merge Conflicts

For the first, imagine that the following happens:

Merge conflict
1
2
3
4
5
$ git pull
Auto-merging index.html
CONFLICT (content): Merge conflict in index.html
Automatic merge failed; fix conflicts and then commit the result.
$

So, what happened? First, you changed and commited some lines in index.html, and during that time, someone else changed something in index.html, too. Second, the changes could not be automatically merged, which means either someone deleted lines so the correct place to put your changes could not be found, or the both of you changed the same line(s).

Now, how you resolve it depends on what you want to do. If you are sure that your changes are correct and should override the other (remote) changes, then you go

Merge conflict - Keeping ours
1
$ git checkout --ours -- index.html

If on the other hand you want to keep their changes and throw away yours, you go

Merge conflict - Keeping theirs
1
$ git checkout --theirs -- index.html

But if you want to really merge the change, that means you keep a little bit of your changes AND a little bit of their changes, then you should use a merge tool.

A merge tool is a (usually graphical) user interface that lets you direct the merge. I prefer to use kdiff3. It has a nice interface and a lot of options, and it is very smart about automatically resolving more complex merges before presenting you an unresolvable conflict. You might want to make yourself familiar with the tool, though, because the multitude of options or buttons might overwhelm you at first.

On invocation, this should look similar to the following:

Merge conflict - using mergetool
1
2
3
4
5
6
7
$ git mergetool
Merging the files: index.html

Normal merge conflict for 'index.html':
  {local}: modified
  {remote}: modified
Hit return to start merge resolution tool (kdiff3):

kdiff3 will try to resolve the conflict once again, and it does succeed every once in a while - then the GUI will never be started. To see what the GUI looks like, you’ll find screenshots at the homepage linked above. git mergetool will leave files with the suffix ‘*.orig’ behind which should never be added to the git index, but can be deleted safely after committing the merge.

Another method would be to resolve the conflict manually. The content of a file will look similar to this:

Merge conflict - conflict markers
1
2
3
4
5
6
7
<head>
<<<<<<< HEAD
    <title>This is the most awesome title</title>
=======
    <title>This is the awesomest title</title>
>>>>>>> 093420abc897324dfe
  </head>

The things <<<<<<, ====== and >>>>>> are called “conflict markers”. These show you what the others (between the first marker and the middle marker) and you (between the middle and the last marker) have added that lead to a conflict. To resolve this, you remove the markers and the wrong change - or you change it to acommodate both, e.g.

Merge conflict - manually resolved
1
2
3
<head>
    <title>This is the most awesomest title</title>
  </head>

After that, you have to commit the merge. Usually, it is sufficient to type

Merge conflict - committing
1
$ git commit

which will open an editor with the most important information already filled out (like an automated commit message, and which files had a conflict).

After that, you are free to push, because the conflict has been resolved.

Overwriting local changes

Now, every now and then, you will encounter a different beast. On a git pull or git rebase, you will see:

Merge conflict - local changes
1
2
3
4
$ git merge
...error: Your local changes to the following files would be overwritten by merge:
    index.html
Please, commit your changes or stash them before you can merge.

This is actually easy and has four resolutions: first, you can just commit your change and maybe run into a merge conflict, which you just learned how to resolve.

Second, you could stash the current work, which means it would be moved out of the way and saved locally, before merging

Merge conflict - stash
1
2
3
$ git stash
Saved working directory and index state WIP on master: 056fdb6123 Add index
HEAD is now at 056fdb6123 Add index

Then you can pull or merge normally, and then apply your local changes again:

Merge conflict - stash pop
1
2
3
$ git stash pop
....
Dropped refs/stash@{0} (...)

This also might lead to a merge conflict - but you know now how to handle these anyways.

The third option is to reset your changes via checkout:

Merge conflict - checkout resetting
1
$ git checkout -- index.html

WARNING! This will throw away the local changes in this file irrevocably!

And the third option would be to reset the source tree before pulling:

Merge conflict - checkout resetting
1
$ git reset --hard HEAD

BIG WARNING! This will reset all changes in all files locally to the last committed version! Your changes will be gone for good!

Push Errors

There’s actually only one: you try to push and it doesn’t work, telling you about conflicts. Now, that’s easy:

What happened:

Push Errors - rejected non-fast forward
1
2
3
4
5
6
$ git push
 ! [rejected]        master -> master (non-fast forward)
error: failed to push some refs to 'git@github.com:....git'
To prevent you from losing history, non-fast-forward updates were rejected
Merge the remote changes before pushing again.  See the 'non-fast forward'
section of 'git push --help' for details.

The resolution is simple: use git pull to receive the changes that you don’t have yet locally. If a merge conflict occurs, go back to the beginning of the document.

Alright, this should cover the most common error messages. Keep cool and remember:

Use git commit before git pull and git pull before git push. That should halve the errors.

TL;DR

If you encounter the following git errors, try:

  1. Merge conflict:

    • git mergetool or
    • git checkout --(ours|theirs) -- filename or
    • resolve by hand
  2. Avoiding overwriting local changes:

    • git commit before pull
    • git stash before pull, git stash pop after pull
    • git checkout -- filename or git reset --hard HEAD to reset
  3. Push conflict: - git pull between commit and push

Questions, suggestions? Try the links on the sidebar or leave a comment.

Happy developing!

IT Today: Sexism, Mysogyny and Frustration

Today, I am pretty much frustrated. It’s quite late at night, and I am coming from a great ruby meet up in Bratislava. Returning with the train and checking for new tweets, I noticed a few posts in the feed that reminded me on discussions I had lately - cumulating in last night. The gist of the things can be found in the post by Jason Stirk (@j_stirk). Please read up on it, and follow the links he suggests.

But what really frustrated me in the end happened only yesterday. I met with a guy I know who also works in IT. And for some reason or other the talk turned to one of the topics I care greatly about: Sexism and Mysogyny in our field of work.

Now, people who know me are aware that I am no saint. I do say politically incorrect things. I do make inappropriate jokes now and again. I am also pretty much fed up with it, and try to do it less and less. I do definitely not need anyones absolution about that.

I try to make amends. Reflect on my attitude and shit I say. I try to help with great events like rails girls or pyladies. I am actually glad that people give me the opportunity to try and make a difference.

I am also trying to raise awareness about the gross mysogyny and sexism in IT whenever appropriate - and maybe too little. Still that does not absolve me.

But that does not mean that I should sit there and hear someone telling me with a condescending tone that ‘misandry exists as much as misogyny and that I should stop overreacting’ - and take it. Least of all when coming from a smart and educated person.

The fuck am I overreacting! I am reacting to your fucking straw man argument that dismisses that people don’t attend conferences because they are feeling unsafe - and you dismiss it on your fucking gut feeling!

You know what? I don’t give a shit about your gut feeling. @j_strike has it right. If you felt threatened and out of your depth in a normal social situation just because of your gender, then maybe, maybe that might give you the right to speak up.

But it will never ever give you the right to tell people that you would only see validity in the mysogyny claims if the misandry claims are given the same audience and attention.

You know why? Because there is not “as much misandry as misogyny going on all the time”. That’s bullshit! What is actually happening is that the very moment any chav will say something about how much the “fucking bitches” screw with this life, won’t most of of his buddies agree? Open your eyes: misandry is abhorred and beaten back with all might by the society - while misogyny is given the slip. It’s pretty much a trivial offense. Oh, it’s just boy’s talk. Lighten up. Don’t take it personally.

I could puke if I heard one of these comments again.

I do have the sneaking suspicion that it’s not even the blatant sexist cavemen that are fast to detest anything feminine that is not bowing to their will immediately that keeps the status quo.

It’s the condescending, talking-down-to-others, mediocrities like you. Those that are always covering up the situation and their own asses quickly. You are part of the molasses that keep our society from moving forward. You with your outdated opinions and your offensive rudeness that you mistake for manners.

You know, I really hope that at one point you will recognize that you have been wrong. And that you will join me in working actively against that horrible status quo that keeps us back in dark ages. I just fear that it’s more likely that you will cling to your twisted view with desperate strength. So I will feel like instead of making a difference, I am losing ground. Again. Like I just did yesterday and today. And again. And again …

But you know what?

Frustration leads to anger.

Anger leads to energy.

And I promise one thing: until I am personally convinced that the society and especially the IT (geek, nerd, …) community as a whole manages to remove the thoughts and words that make people think and talk of “us” and “them” entirely - until then I will fight for what I think is right and the minimum of decency and decorum. With all I have.

You will not stop me there. I dare you to try.

Recommended Readings 2013-01-16

Here’s a short list of themes around OOP and Ruby that I would recommend:

Practicing Ruby - Building Enumerable and Enumerator in Ruby
Makes a very well founded, simplificated introduction in Enumerable and Enumerator with great examples - concentrating on the major points.

Binstock on Software - Perfecting OO’s Small Classes and Short Methods
Runs you through really hard restrictions to look upon your classes.

Goodbye, shitty Car extends Vehicle object-orientation tutorial
A good post about not using “Real World Objects” for explaining OOP and polymorphism.

Test-Driven Development. A Cognitive Justification? - Steve Freeman
A critical and still valid view on TDD and Pairing.

Management Myths Debunked by Johanna Rothman

Just wow. This morning’s lecture, brought to yours truly by Johanna Rothman was a joy to read. Starting at an article brought to my phone via Google+ I could not stop reading the referenced article.

And then I dove into the whole series dubbed “Johanna’s Management Myths”, namely:

  • The Myth of 100% Utilization
  • Only the ‘Expert’ Can Perform This Work
  • We Must Treat Everyone the Same Way
  • I Don’t Need One-on-ones
  • We Must Have an Objective Ranking System
  • I Can Save Everyone
  • I Am Too Valuable to Take a Vacation
  • I Can Still Do Significant Technical Work
  • We Have No Time for Training
  • I Can Measure the Work by the Time People Spend at Work

What impressed me most is the things those articles have in common. First, they are easily understandable by anyone who does or did any kind of management work. Then, they are to the point. And you do have the feeling that each and every article has not been brewed up by a person sitting in an ivory tower, but by someone who did live through all the things herself. Add to that a real talent for writing and explaining things, and you know why I liked it.

Also, I can relate to those things. 100% Utilization? Measure Work by Time at Work? I was working for a certain person, a former two-man show turning company with all the obvious growing pains. The “big boss” managed to bully almost all employees into working more than the regular 38.5 hours (Austrian employment laws rock). Additionally, if someone was having a chat with someone else but the boss, he basically went from Jekyll to Hide in less than a New York second.

The only thing that he created was a working environment where most of the people left after a few months. And he was constantly complaining that he couldn’t keep people. They were all out after his money and would never put in enough effort etc pp.

Some other traps like not planning enough time for training, being too valuable for vacation, trying to ‘save’ everyone where traps I also fell into myself. Others, like not having time for One-on-ones I have to admit I am guilty right now. I am going to change that beginning today, though.

Having been working in agile environments, helping to create them (also currently) at least relieves me from treating everyone the same way, or having “experts” on the team - or if it happens, we’re removing those impediments fast.

While in former employment situations, having a boss that micro-managed by doing programming stuff without having the understanding anymore was a problem, this has never been true for my current employer. Everyone is trusted to do her/his best and mistakes are learning opportunities. I myself am still a programmer and servant leader, so me doing programming is the norm, and well supported by the team, so personally I just do what I am supposed to.

I just wish I would have reacted like the manager in this situation. I really should have. But I can try to avoid a situation like that myself if I would ever be in this position.

Well, that sums it up: Easy to read and understand, real great work. Thank you, Johanna Rothman, for that valuable insight on those complex topics!

Burnout Is Real

Yes, it’s true. Debating about semantics like “it’s not in the ICD-10” is not helpful.

Disclaimer: I am no psychologist or psychiatrist. All of the following things are conclusions based on personal experience.

Let’s go through some myths:

Myth 1: Burnout does not exist

WRONG. As of now, the word “burnout” is a fashionable description of various syndroms footing in troubles all around work, recognition and work-life balance.

But: At the root of a so-called “burnout” is usually a multi-faceted depression syndrome. Just because there’s not standardization there is no accounting for a common word describing a well-known and ill-felt phenomenon.

Problem: While in a lot of states receiving psychological support is slowly being accepted, depression still has a lot of social stigmata and misunderstandings attached which keeps people suffering from depression from searching much needed help. And while calling it something else might superficially lift the stigma, in reality it does not. The next paragraphs will deal a lot with ways that burnout is being marginalized, scoffed at and reduced to one dimension.

Myth 2: Depression (or burnout) is mainly self-inflicted

WRONG. That’s about as smart as saying that cancer is self-inflicted. There might be factors that can be influenced by yourself, but not everything can be controlled just by say-so.

Problem: As long as this prejudice exists, people suffering from depressions/burnout are less likely to search help, but will feel like it is their fault alone, thus adding to the depression. A very dangerous vicious cycle!

Myth 3: People suffering from burnout can just work a little less to avoid it

WRONG. As with every depression or serious clinical behavioural state, the root of a depression and the immediate triggers are usually not the same. And the roots can be many, like missing respect or recognition, an overwhelming yet often blown out of proportions feeling of responsibility for co-workers, projects and the success of the whole company,

Problem: A sudden drop of workload will grow the depression. You will feel worthless and not up to the tasks she is responsible for. This will be reinforced by superficial acknowledgment of a “problem” by the colleagues paired with misunderstandings, thus forcing the depression into a new down.

Myth 4: Only managers/politicians/IT workers can suffer from burnout

WRONG. Depression is more likely to be detected and taken seriously in certain social circles. This does not mean that the rest of the world is free of it - just that it usually does not receive the treatment necessary.

Problem: See above. Not treating a depression does not mean it doesn’t exist at all. It’s just an elaborate form of denial.

Myth 5: If you suffer from burnout, you hate your job, and just have to find one you love doing

WRONG, WRONG, WRONG. Especially if you are very good at what you are doing for a living and feel responsible for every teensy bit of it, you are most likely to carry on during your free time, evenings and weekends alike. Not being able to find a healthy work/life balance is often key to a depression where you feel undervalued and misunderstood along with the strong feeling of being inadequate.

Problem: In our working world it is envied to be able to do what you love to do for a living. So, young programmers create software all day at work, only to do the same over night and in the weekend. Even if that satisfies them they will come to a point when they feel either their success at work or their success at their private projects is lacking, thus preparing the grounds for a depression. Unproven, but seen from personal experience: if you dislike most aspects of a job you are doing, it is easier to divide work from the rest of your life, and thus avoid a lot of depressing thoughts, a la: “it doesn’t work as well as it could? Hard luck, I’m not going to dive in and fix it.”

Myth 6: Everyone’s a burnout candidate - you can’t avoid it

WRONG: A short half-truth. Yes, in the “right” setting, almost everyone will at one point feel struck down and totally overwhelmed. If they are in a responsible work relationship and their managers/bosses/employers/whatever see the problem early enough, plans can be made to manage the workload.

Problem: Especially in customer-faced complex work or self-employment, the necessary screening via superiors is ofttimes missing - same with unexperienced managers etc. It needs a lot of understanding and awareness. If this can be achieved, a burnout can ofttimes be avoided.

Myth 7: Burnout is just a result of bad time management

WRONG: Bad time management, either brought on by yourself or your organization/boss/… is never the solitary cause of a depression. Yes, it can contribute to it, but only changing how the workload can be managed in a timely fashion rarely does change feelings about inadequacy and underappreciation.

Problem: Time management seminars and books have found a new lever: burnout as the evil witch to be burnt at the stake with better time management as the fuel. This can work in some cases and will not work in most. It will, however, smoothen any process towards a healthier work/life balance.

So, what to do?

There is not one single working recipe. There are a lot of things that can help, though.

For managers, it helps to watch your employees and help them to balance their workload. If you are building a business, having sustainable pace is more important than getting the stuff out right now.

For co-workers: try to not get too involved. Offer help if there is obvious signs like listlessness or stalling - but only take on what does not burden yourself. Sharing work is often a good start and helps both, reciprocating on that might turn into a blessed circle. Just be aware of your own work/life balance and don’t get burdened with responsibility for others overly much. It’s a really narrow path to walk, be aware.

For affected: lighten the work load, but find something else to put your energy into (dancing, hiking, biking, gym, …). Go seek professional help. And don’t listen to people that tell you that it’s “just” …anything. It’s never “just” an imagination, “just” working too much, “just” bad time management … it’s about the way you feel about yourself, towards others and your responsibilities. Maybe your coping mechanisms are not up to the task, but that can be trained as well as any other thing.

Don’t be disheartened. I won’t say your path is trivial and easy, because it’s not. And you will not necessarily become stronger by the experience. But you might become wiser after living through it.

The Scrum and Kanban Rant Rant

What the heck is up with all the rants around Scrum and Kanban (and Agile in general)? I know, it’s a fading fad, but really, is all the bias and negativity necessary?

Who fights whom?

I chose my words carefully. Not just lately, an endless fight is emerging, and people are squabbling over semantics, technicalities and bask in their own ignorance. But do the agile enthusiasts and evangelists fight the PMA/I or V-Model crowds? No, the quarrel is endlessly growing between different movements that are based on the same ideas: the agile manifesto.

With seemingly endless energy people do all but stop short of denigrating, namecalling and simple misrepresenting others or ideas that differ from their own experiences.

What’s the goal?

I actually have no idea. I just can assume that people are trying to make themselves heard in order to enable others to share their opinion. This is at least how I would like to do it. Though I fear that a lot of people who are posting do have less lofty goals in mind. Usually people ranting and raving about “the other” methods (and methodologies) seem to be consultants, trainers or somehow part of an organization that tries to generate revenue by jumping on the agile train.

So, lofty goals or just plain trying to bang up the buck? Decide for yourselves.

What’s all the fuzz about?

For me it seems that mostly lately people accuse each other of using the “wrong” methodology. TDD vs BDD, Scrum vs Kanban, the list is as endless as the discussions are. I want to dissect a few argument chains now and underline the points that aggravate me:

Scrum is bad, Kanban is better

There is nothing like that. There are situations where Scrum is more easy applied than Kanban, and vice versa. Having a tight knit team and a struggling organization can be as easily solved by either as having a dysfunctional team and a strong organization. The approach though is very different, and it depends on the individuals involved as well as the current mood and structure of team and organization as to which methodology can be applied more easily.

Some organizations need to be shocked into awareness and collaboration: Scrum can help there. Some organizations need to be eased into change: Kanban might be the better choice.

All in all, use what works better and what people can accept at the moment. Don’t be afraid of changing the decision after a while.

“We are a (Scrum|Kanban) company” is pure poison. It limits the choices as much as an “We are an (IBM|MS) shop” or “We program only in (C++|Java|Python|Ruby)”. Don’t go there. It’s much harder to come back from that later. Say “We are using whatever the people and situation demands”. Repeat it 300 times. Go to sleep. Dream well.

Kanban cannot increase productivity as Scrum can

Who says that? A few disgruntled people that have ben rubbed the wrong way? Cassandras that seemingly fight against an onslaught of ignorance emanating from the rest of the world? From my experience it’s two groups: developers which have been badly burnt by well-meaning and worse-doing consultants or Kanban enthusiasts that take the enthusiasm a bit too far. The latter usually spew lines of “Scrum is too restrictive” or “timeboxing is bad” without showing the least understanding of the “Why?”. The former are to pity - they fell into exactly the trap that the consultants laid out for their bosses (and their money). Those consultants smear honey in form of “increased productivitiy” and “diminishing costs” around the deciders’ mouths.

What then happens is pretty typical: because of the motivation of change and the promise of a better future, the team and lots of others in the organization work more. Thus the productivity increases by staggering numbers, even factors. And when the people are dropping like flies after being burnt out, the consultant has already received his money and moved on …

Note: Increased productivity is and never has been a goal of agile development, nor is it part of the agile manifesto. It is a welcome side-effect if all the issues that kept the team from performing has been removed or can be controlled by the agile method applied. Every consultant that promises increased productivity by any agile method being more than marginal from the beginning is a charlatan, a snake oil seller of the worst kind and should be tarred, feathered and flogged beyond the city limits.

Kanban is scientifically based, Scrum is not

That’s actually brilliant: a hard selling point. But if you happen to be close enough, you will see one thing: Most companies who claim to use Kanban are not using one of the proclaimed scientific methods to be able to have sustainability and measureability. At least not longer as their current Kanban consultant forces them to do it. Rarely people understand the value and gain of perpetual, scientific measurements, analysis and reporting. So they don’t do it. What’s left? Daily Standups (that are not even part of Kanban as a methodology) a board, and WIP limits (that are circumvented more often than not).

And what’s left over of Scrum after a while: Daily Standups, a board and estimates (if even). Sounds familiar, does it?

Timeboxing is bad, Kanban has no such limits

First: timeboxing, judiciously applied, can be a life saver. Timeboxing also means: partitionate your work into manageable bits which can fit into the selected timebox. Have clear goals and committments before you start to work. It does not mean that you have to cram everything remotely related to the current problem into the available timeslot.

An example? Sure: various teams use the “timebox” of a standup meeting as a means to curtail discussions and complaints. If the allotted time has run out, further input is strongly discouraged and sometimes even prohibited. For us, it was way more useful to do what Scrum (and others) suggest: our Daily Standup has the goal to keep everyone informed of progress, problems and next efforts. We are doing it standing up and keep it as short as possible and within the team.

Every other problem is discussed afterwards. Often all team members and others of the company are involved, but even if we keep it short and in the same area than the standup, it is made clear that the standup is over at that point. Abundantly so. People who can or do not want to participate will leave. And at the beginning of that improptu discussion, two things are made clear again: the goals and the allotted time. As all meetings should be.

Kanban and Scrum may not be mixed

Who the fuck said that? Either again a money grubbing conslutant or someone who has no two brain cells to interact. At the very core of both of those methodologies is the feedback loop. Which is ideally an application of the Deming cycle.

Repeat with me: 1. Plan 2. Do 3. Check 4. Act

Go back to 1. and repeat again. And again. And again. Got that now?

So, what does that mean for mixing concepts? Please do it. If you need a WIP limit on any column of a scrum board - implement it. If you want to work more pull driven on your scrum board? Do it. If you want to note worked team hours on your scrum tasks? Do it already!

Don’t be fools, use all your tools.

In the end, every methodology is a tool. It’s never wrong to use them for the right purpose. If you use the right tool in the right environment, it will work. If you use the wrong … well, you get it.

You just have to know your tools, and mostly it seems that people don’t take the time or experiment enough to know the tools well enough to use them. This is dangerous in power tools as well as in agile tools.

Some postulate that using Scrum or Kanban should be done purely - don’t mix and match, don’t let other experiences distract you.

This is true. Especially when you or the organization is new to the whole thing, it will be good to have some standard literature to read up on, and experiences within known parameters to share with others. If you start to adapt early without really checking what and why you are adapting, the experiment has a big chance to fail horribly.

Some postulate that using Scrum and Kanban purely should be refrained from - pick the best from every method, and everything will just work!

This is also true. It does not hurt to adapt the methodology judiciously after knowing what and why to adapt. But just mixing and matching because you feel uncomfortable with some detail without investigating why you feel uncomfortable about it is against common sense.

Also, the things you can change with Kanban are very few: you can select what to measure and what to adapt to, which goals and warning limits you set - but after all, this is well within what Kanban is about.

With Scrum, it’s easier to break stuff - abolish meetings that would be necessary just because people don’t feel good about it, softening up sprints and sprint goals - that just points to underlying problems, and complying with that feeling will just bring those things to a grinding halt.

In the end, it’s like software testing: a little is better than nothing, but nothing can beat to use all of it.

And what about all the others that are saying that I am wrong?

You mean, all the others that are squabbling and loudmouthing instead of pursuing the common goal of creating better work environments and sustainable paces, as well as great and working products that do just what the customer needs?

Screw them. In the middle of your brain, you know better. Trust your feelings. And just do what you do best: deliver great products with the gut feeling that you do it the right way.

Stylish Stylings

We haz new stylez!

Thanks to @fadr, there’s a lot less pain when reading this blag.

Told you, I am no designer. Good to have a good one around!

Datamapper: Customizing Associations

Datamapper is an alternative ORM for various ruby projects, and can work as an ORM replacement for ActiveRecord in Ruby on Rails. The most pervasive arguments for using Datamapper over AR is the easier handling of legacy databases and the intelligent deconstruction of N+1 queries, and the fact that functionality is splitted into small plugins, so for simple projects it does not weigh down the framework codebase with unused code.

One other strength for me is the auto-migration part: You do not have (but you are able) to add separate migrations that pretty much reflect what you have configured in the models already. It takes the configuration of the model properties and creates the tables from that (destructive, non-destructive: your choice). This means that the control of the properties of a data model is no longer handled by a separate script, but by the definitions of the structure of the models itself. This can optionally include referential integrity in the underlying database, too.

Also, the documentation is really good. There are whole chapters handling understanding of HABTM (has n, through: …, self-referential HABTM relationships et al). But one chapter left me stumped a bit:

How to customize a relationship?

Maybe it was me skimming over the surface of the documentation, maybe my current needs were not exactly covered, but reading the fine manual was not quite enough for me. I still managed to create classes that wouldn’t play nice.

The problem:

I had a simple model, User:

User Model - user.rb
1
2
3
4
5
6
7
8
class User
  include DataMapper::Resource

  property id, Serial

  property name, required: true, length: 3..255

end

And another simple model, Message:

Message Model - message.rb
1
2
3
4
5
6
7
8
9
class Message

  include DataMapper::Resource

  property id, Serial

  property subject, required: true, length: 3..255
  property text, lazy: false
end

Now, I wanted to add a relation from User to Message as follows:

  • a User has many sent messages
  • a Message belongs to one sender

Doing it as a normal relation would be simple, as follows:

First, I change the User model:

User Model - user.rb
1
2
3
4
5
6
7
8
9
10
class User
  include DataMapper::Resource

  property id, Serial

  property name, required: true, length: 3..255

  has n, :messages

end

Then the Message model:

Message Model - message.rb
1
2
3
4
5
6
7
8
9
10
11
class Message

  include DataMapper::Resource

  property id, Serial

  property subject, required: true, length: 3..255
  property text, lazy: false

  belongs_to :user
end

So far, so easy. Now the more complex part: I want to have another, more complex relation:

  • a Message can have many recipients
  • a User can have many received messages

Now, as you can see immediately, this is going to be a real HABTM-relationship. But when I started to add the relation to the User model, I saw oddities immediately:

User Model - user.rb
1
2
3
4
5
6
7
8
9
10
11
12
class User
  include DataMapper::Resource

  property id, Serial

  property name, required: true, length: 3..255

  has n, :messages

  has n, :messages, through: Resource  # That will not do.

end

At this point I decided to just rename the first relation to what I already suggested: sender and sent_messages:

User Model - user.rb
1
2
3
4
5
6
7
8
9
10
11
12
class User
  include DataMapper::Resource

  property id, Serial

  property name, required: true, length: 3..255

  has n, :sent_messages, 'Message', child_key: ['sender_id']

  has n, :messages, through: Resource

end
Message Model - message.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
class Message

  include DataMapper::Resource

  property id, Serial

  property subject, required: true, length: 3..255
  property text, lazy: false

  belongs_to :sender, 'User', child_key: ['sender_id']

  has n, :users, through: Resource
end

As an explanation: If I rename a relation, but there is no model that corrensponds to the singularized name (e.g. Sender), Datamapper will complain when migrating or instantiating the relationship. So, I have to tell datamapper to use the ‘Message’ model for the relation ‘sent_messages’, and the ‘User’ model for the ‘sender’ relation. The constraint ‘child_key’ helps with selecting the correct foreign key field in the child model - the side that has the ‘belongs_to’ association added.

And the second association will add a table messages_users with the fields user_id and message_id as primary key. This should be sufficient …

But no, I wanted a bit more: first, I wanted to store the time the message was read by each recipient, and then I wanted to have nice names for the associations without using (ugly) proxy methods.

The first part is easy: I will add another model, Notification, that keeps the recipient/message relation and has a ‘read’ DateTime field as payload:

Notification Model - notification.rb
1
2
3
4
5
6
7
8
9
class Notification

  include DataMapper::Resource

  property :read, DateTime, required: false #if it is null, the associated message has not been read yet

  belongs_to :message, key: true
  belongs_to :user, key: true
end

The key: true constraint on the property means that both fields are part of the primary key. Now let’s adapt the other two classes. User first:

User Model - user.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
class User
  include DataMapper::Resource

  property id, Serial

  property name, required: true, length: 3..255

  has n, :sent_messages, 'Message', child_key: ['sender_id']

  has n, :notifications # this is a simple has many/has one relationship

  has n, :messages, through: :notifications

end

Message second:

Message Model - message.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class Message

  include DataMapper::Resource

  property id, Serial

  property subject, required: true, length: 3..255
  property text, lazy: false

  belongs_to :sender, 'User', child_key: ['sender_id']

  has n, :notifications # this is a simple has many/has one relationship

  has n, :users, through: :notifications
end

The through: constraint tells Datamapper to use the notifications table as underlying n:m relationship table.

But now I wanted to make it really neat: from Clean Code we know that having methods that are named for their purpose create more value that cryptically named ones. So, I decided to rename those associations to reflect theirs:

User Model - user.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
class User
  include DataMapper::Resource

  property id, Serial

  property name, required: true, length: 3..255

  has n, :sent_messages, 'Message', child_key: ['sender_id']

  has n, :notifications # this is a simple has many/has one relationship

  has n, :received_messages, 'Message', through: :notifications

end
Message Model - message.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class Message

  include DataMapper::Resource

  property id, Serial

  property subject, required: true, length: 3..255
  property text, lazy: false

  belongs_to :sender, 'User', child_key: ['sender_id']

  has n, :notifications # this is a simple has many/has one relationship

  has n, :recipients, 'User', through: :notifications
end

… to find out that this does not work.

First, the notification tables looked like this:

1
2
3
4
5
Notifications:
  user_id             | Integer
  recipient_id        | Integer
  message_id          | Integer
  received_message_id | Integer

Trying to use the models on the console lead to messages like:

property or constraint recipient not found on model User

Well. Erm. No good. But there is a solution already built in, but the documentation all but skips it:

[…]we will use :via to be able to provide “better” names[…]

Solution:

But the rest of the example took me a while to figure out. I will let the finished example speak for itself:

User Model - user.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
class User
  include DataMapper::Resource

  property id, Serial

  property name, required: true, length: 3..255

  has n, :sent_messages, 'Message', child_key: ['sender_id']

  has n, :notifications # this is a simple has many/has one relationship

  has n, :received_messages, 'Message', through: :notifications, via: :message

end
Message Model - message.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class Message

  include DataMapper::Resource

  property id, Serial

  property subject, required: true, length: 3..255
  property text, lazy: false

  belongs_to :sender, 'User', child_key: ['sender_id']

  has n, :notifications # this is a simple has many/has one relationship

  has n, :recipients, 'User', through: :notifications, via: :user
end
Notification Model - notification.rb
1
2
3
4
5
6
7
8
9
class Notification

  include DataMapper::Resource

  property :read, DateTime, required: false #if it is null, the associated message has not been read yet

  belongs_to :message, key: true
  belongs_to :user, key: true
end

So, what does :via tell Datamapper? Fairly simple: it says: ” The reverse relation of this relationship is called “:message” in the “Notification” model”.

Reading the code at the documentation example first, I tried the following:

  1. Let :via point to the reverse relation in the other model (User or Message, not Notification)
  2. Let the :notification relationship know that the child key is :message_id for :received_messages in Notification (and :user_id for :recipients in Message)
  3. Both of the above 4. …

It took me the better part of two hours to read through the code and realize where I made the wrong assumption. But now everything works.

Conclusio

If you want to customize a HABTM-Association in Datamapper, make sure that:

  1. The link model (here: Notification) uses the derived name for the association as is standard: “User” is related by “belongs_to :user”, “Message” by “belongs_to :message” and so on.

  2. The other models’ relations know what the relationship’s reverse is named: In the “User” model, “received_messages” is defined as

    has n, :received_messages, 'Message', through: :notifications, via: :message

    where

    • “:received_messages” is the relationship name
    • “‘Message’” the name of the model that is linked via this relationship, and
    • “via: :message” denotes the reverse of the association in the “link” model (here: “Notification”).

I hope that this helps someone to save some time and helps understanding.