UI Automation Testing Before Deployment
My question is focused on if my setup is currently following a best practice approach regarding BDD with UI acceptance testing. I'm using WatiN with SpecFlow to build my UI acceptance tests and I understand upon deployment AppHarbor runs all unit/integration tests and only pushes the site live if the tests pass. So I started by first writing a basic failing test as follows:
Scenario: Navigation to homepage
When I navigate to /
Then I should be on the application homepage
The steps associated with this test open up a browser using WatiN and verify the title attribute of the view is set to "Welcome". I am doing a check on the environment to decide what URL to test against with the WatiN browser. For example, if in development navigate to "http://localhost:49641/" for home. Otherwise, navigate to "http://myappharborapp.com/".
My problem is that if you are deploying this application for the first time, the page or view actually does not exist and so the test fails (because the site is not live yet). This would also fail if for instance I would later add an "About" page view and wrote a failing test first. When I push the updates the test will fail because the "About" page does not yet exist.
My question then is: Am I not following best practices with regards to how you should be setting up your UI tests? How should these tests be setup so that they pass in any environment?
Any insight is greatly appreciated!
Discussions are closed to public comments.
If you need help with AppHarbor please
start a new discussion.
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
Support Staff 1 Posted by tt on 03 Feb, 2011 10:50 PM
We don't currently support UI testing.
We always test the code before deploying it, so you will never (as with our current design) be able to test against the website. You can do it after the first deploy, but even at that point, you would test the currently deployed versions, which may cause a test to fail and thus make you unable to ever deploy a new version.
We will be evaluating ways to spin up a temporary web server after the build, that tests can target. This is not something on top of our list though.
I recommend that you place your acceptance tests in another project than your unit tests. If you do so, you can create a special solution called "AppHarbor.sln", which we will then use for the build, and which doesn't contain the acceptance tests.
tt closed this discussion on 03 Feb, 2011 10:50 PM.