🏢 Deployments
Effective test environments should replicate the production environment for the most accurate testing:
🏠 Local Deployments
- There are several advantages to having your own Deployment:
- Cypress E2E testing - the
data-cyattribute strategy can be deployed on elements/components to test easily.- This strategy has the advantage of decreasing test maintenance since updates to an application may remove or alter DOM locators you were previously relying on to test. The
data-cyattribute strategy both tells the developer to keep the attribute wherever the element/component is moved while ensuring your tests run reliably.
- This strategy has the advantage of decreasing test maintenance since updates to an application may remove or alter DOM locators you were previously relying on to test. The
- Cypress E2E testing - the
🔬 Testing Servers
- Sometimes there are aspects of testing an application on a VPS-hosted server that cannot be properly replicated locally or otherwise.
- A local deployment may not run properly without some tweaking - depending on your operating system - but will run properly on an AWS instance for example.
- Even when it comes to docker containers, there are situations where an Ubuntu host will need a slightly different configuration than a MacOS or Windows host. (Regarding Docker, usually the OS-dependent configuration differences are fixed in
docker-compose.ymlor sometimes theDockerfile.)
- Even when it comes to docker containers, there are situations where an Ubuntu host will need a slightly different configuration than a MacOS or Windows host. (Regarding Docker, usually the OS-dependent configuration differences are fixed in
- A local deployment may not run properly without some tweaking - depending on your operating system - but will run properly on an AWS instance for example.
🏭 Production Servers
- QA should be as careful as possible not to break anything on production.
- Any potential tests should be performed locally or on testing servers, but there are some instances where testing on Production is required.
- In these cases QA should always think of ways to identify/declare the test as a "test" as much as possible.
- For example in creating a user on Production the user should be named "Test User" or a test order for an e-commerce site should have a Name/Label/Notes which declare the payload is part of a test. Often in online ordering the Comment/Instruction should be "TEST ORDER - DO NOT MAKE".
- These tests, once verified and documented should likely have any remaining data removed.
- In these cases QA should always think of ways to identify/declare the test as a "test" as much as possible.
- Any potential tests should be performed locally or on testing servers, but there are some instances where testing on Production is required.
❗ Testing on Production
- Production tests should be limited to lower-risk, happy-path and specific integration-oriented concerns that cannot be properly replicated on other testing environments.
- Examples:
- Low risk - Modifying, adding or removing items for a test account on production.
- Happy-path - Using the application on production in the way it was intended without attempting to find errors or edge cases
- This is especially important with regard to any testing which could affect the database.
- Specific integration - Often when production gets a new release with a new API integration this needs to be tested to ensure the appropriate keys are valid and the functionality matches prior testing.
- Terminology:
- An often-used term/lingo in testing on production is called "Dog-Fooding" whereby the internal development team/sales/QA/designers will use the application they built in-house for awhile to see if any defects are discovered. "Eating your own dog food.." as the expression goes.
- As strange as it may sound sometimes there are still bugs on Production despite the team's best efforts to replicate the environment as much as possible.
- Why?
- More active concurrent users
- More data transfers
- More languages/locales
- 3rd party API key/integration errors when the same vendor's test-keys work properly.
- Why?
- Examples:
ℹ️ Considerations
- Local vs Hosted
- Permissions & Authorization
- Basic Auth
- Sometimes a basic auth login will prevent some features from functioning properly. For example Progressive Web Applications (PWAs) will not recognize the session if it is behind basic auth. To get around this, QA may have to request that basic auth be disabled for these tests.
- Office network firewalls or Adblocking DNS Sinkholes
- Sometimes when needing to test tracking cookies, analytics or other advertising-SEO-related functionality, it becomes necessary to disable your browser or home network adblockers/content filters.
- Basic Auth
- Network Replication
- In some scenarios where an application would only be used when connected to mobile networks and GPS systems such as Google Maps or other applications related to being outside of a LAN, the connection speeds and/or intermittently-connected nature of the service becomes enecessary to replicate.
- For example in testing a GPS application, QA should enable geolocation and either spoof various locations for testing or use a real device locally.
- Another example would be if the connections speeds are slower in more rural areas, QA should test the user experience when data takes longer to transfer. When this is relevant and neglected an application may become utterly unusable.
- In some scenarios where an application would only be used when connected to mobile networks and GPS systems such as Google Maps or other applications related to being outside of a LAN, the connection speeds and/or intermittently-connected nature of the service becomes enecessary to replicate.