Software Testing

Available on: http://pokoli.github.io/software-testing

Who am I?

Sergi Almacellas Abellana

  • (Python ♥) Programmer
  • Open source enthusiast
  • Automation Maniac

Bug Fixing Costs

TDD & BDD to the rescue!!

Test Driven Development

  1. Write Test Case
  2. Ensure Test Fails
  3. Implement Functionality
  4. Ensure Test Pass
  5. Refractor
  6. Ensure Test Pass

TDD Example


def sum(a, b):
    return a + b
                    

self.assertEqual(sum(1, 2), 3)
                    

self.assertIsInstance(sum(1, 2), int)
self.assertNotIsInstance(sum(1.0, 2), int)
                    

self.assertRaises(TypeError, sum('1', 2))
self.assertRaises(TypeError, sum())

But how to tests external services?


import requests
from datetime import date
url = 'http://www.ecb.europa.eu/stats/eurofxref/eurofxref-daily.xml'

class CurrencyConverter(object):

    def get_ratios_xml(date):
        return resquests.get(url)

    def update_ratios():
        self.parse_data(self.get_ratios_xml(date.today()))

    def get_eur_to_usd(self, amount):
        if not self.ratios:
            self.update_ratios()
        return self.ratios['usd'] * amount

Mockups to the rescue!!!


from mock import patch

converter = CurrencyConverter()

def get_local_ratios_xml(d):
    self.assertEqual(d, date.today)
    return open('local.xml').read()


with patch.object(CurrencyConverter, 'get_ratios_xml', new=get_local_ratios_xml):
    self.assertEqual(converter.get_eur_to_usd(1), 1.12)

Behaviour Driven Development

  1. TDD Extension (same filosofy)
  2. Describes User Interaction
  3. Who
  4. What effect
  5. Which bussiness effect

BDD Example

Story: Application login
In order to use the aplication
As a application user
I want to Enter the application
Scenario 1: Login with email and password

Given I have and valid email
And I obtained a password with signup
When I enters the email and password
Then I see my homepage.

Code Coverage


$ coverage report -m
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
my_program.py                20      4    80%   33-35, 39
my_other_module.py           56      6    89%   17-23
-------------------------------------------------------
TOTAL                        76     10
  • Mesure which % of code is tested
  • Allows to know which lines are not tested
  • As HTML report if desired

Please don't go mad with 100% coverage

When to run the tests?

  • Programmer always run tests?
  • Sure?
  • Always always always?
  • I don't belive you

Continuous Integration to the rescue!!!!

Continuous Integration

  1. After commit hook
  2. Notifies author/manager when failure
  3. With multiple targets (e.g. Python 2 vs. Python 3)
  4. With multiple databases (e.g. PostgresSQL vs. MySQL)

But TDD+BDD+CI is enough?

NO!!Of course

Continous delivery to help you!

Continuous Delivery

  • Once CI has passed
  • To a staging server
  • Normally using commit hash
  • URL sent to testing team
  • Cheaper with containers (Docker)
  • Production when tagged

Summary: Full Process

  1. Write Test
  2. Implement Functionality
  3. Run Test
  4. Commit and push
  5. CI Test on all platforms
  6. CD upgrades staging machine
  7. Repeat 1. for each functionality
  8. Testing team says OK
  9. Create tag on repo
  10. Commit and push
  11. CI Test on all platforms
  12. CD upgrades production machine

Summary: Results

  1. Developers
  2. Quality Team
  3. Support Team
  4. Product Manager
  5. Users
  6. Managers

Summary: (Real) Results

  1. More Quality control
  2. Find problems earlier
  3. Prevent regressions
  4. Code documentation
  5. Faster Development
  6. Faster Feedback
  7. Reduce change fear

Bonus: Upgrade dependency libs (personal experience)

Migrations ♥ Tests

My steps to migrate to new version:

  1. Write tests (if none)
  2. Migrate tests (if required)
  3. Update dependencies
  4. Run tests
  5. Fix errors
  6. Repeat untill all green

Thank you!

The presentation code is avaiable on

http://github.com/pokoli/software-testing