test in production

From IndieWeb

5616559901_8ca0186c11_z.jpg

test in production is (AKA testing in production) deploying code changes to a web site or service typically visible to the public as part of checking to see if the code changes function as expected, and is practiced by some IndieWeb creators especially with their own projects as a form of eating their own cooking, often before publishing code changes to an open source repository.

Why

Testing your code in production on your own site is one way to use what you make before sharing it with others to make sure it behaves as expected, and personally take the risk & responsibility for those changes rather than offloading those burdens to whoever might be using your open source project.

Precautions

While testing your code in production can be a good part of using what you make, security precautions should still be taken. Showing errors, warnings and notices usually reserved for dev environments can be a big security risk due to the fact that things like paths, usernames, secret keys, etc. might be inadvertently shown to anyone who cares to look.

How to

How to log errors

It’s not great UX to have confusing error messages intermingled with content.

Rather, you should log all such messages somewhere where only you can see them, or only show them in-page if you’re logged in as an admin.

In PHP, there are several ways to go about this.

  • Setting display_errors off in your php.ini
  • Within PHP: ini_set('display_errors', 'off');
  • For a single line: @codeWhichIsCausingErrors();
  • If you’re logged in as an admin:
    if ($user->isAdmin()) ini_set('display_errors', 'on');

IndieWeb Examples

Tantek

Tantek Γ‡elik has been testing CASSIS in production on his personal site tantek.com since creating it and continues to do so, in the hopes that any errors in code changes will be revealed on his personal site before they affect others.

Manton

Corporate Examples

When corporations (e.g. silos) test in production, the results are often documented in unflattering ways unfortunately.

While "testing in production" may exercise some aspects, it is not a replacement for formal testing & QA methods.

CaUkTFuUcAEDcim.png

@benward: Great moments in code review

(at an unknown corporation)

Other Examples

Solarwind

Button with saying about Test In Production, with smallprint saying Solarwind

See Also