Just uploaded the first part of the presentation of Cataroo.
I already have the screen-recordings of the manual executions of the test-cases. I need to edit them to present of a single, coherent story. I still have to think about a couple of points to say especially in the design of the test-cases section and data analysis section. I might be able to post the narration for the second (hopefully final) part tonight.
At this very moment I'm making a video presentation of a test-tool I just made. It is a cross-breed between that bastard thing, and a great test framework named KitCAT (Kit for Converged Application Test).
What's cataroo, you might ask. Well, here's the story. First, I named it BastardizedCAT (from Bastard + KitCAT). But then I found the name a bit ugly (it doesn't sound funny). A lookup in dictionary gave me this:
bastardized adj. (Arts): deriving from more than one source or style. impure (vs. pure) --- (combined with extraneous elements) [WordNet 1.5].
So, it's cross-breed, sort of. Coincidentally, when I was thinking of a better name, my cat was doing weird poses on my bed (please don't get me wrong). I was like: "Huh? Is that a kangaroo lying on my bed?". That's how the name cataroo came up (a cross-breed between cat and kangaroo).
Alright, here's the storyboard, with narration, of the presentation:
UPDATE: because I wouldn't be able to finish the video tonight, I decided to uploaded the scenario.xml (that contains the automation code) here. Look for it at the end of this posting. This is to give a sneak peek of what is to come....
(1) What's the purpose of this presentation.
In this presentation I'm going to show a way we can automate the testing of converged application. As in my other presentations, I will still be talking about telephony application, specifically the ones that use SIP as the signaling protocol.
(2) What's the story line of this presentation.
Before we get to the point of crafting the automation of our tests, we certainly have to start with an understanding of what we will be testing. Therefore I'm going to start with an explanation of the application I use as the vehicle in this presentation. You can take it as a quick-briefing about the requirement of the application.
Then, based on the understanding, we will move on to defining the test cases. We are not going to go in depth here with the techniques of designing system test-cases for this kind of application. So, we will just settle with a couple of basic / normal use-case scenarios.
Once we have the test-case defined, we, ourselves, will have to run it. This activity is useful, obivously, to validate the test-case. We might learn that we have missed a couple of steps, made several wrong assumptions, etc. Related to the automation, one of the aspect we will have to learn / investigate by manually executing the test-case is the data. At the end of the day, I think, an application is about consuming, processing, and producing data. So, we want to make sure all the elements involved in the scenario that we are testing are producing and getting the correct data, which implies -- to some degree -- it processes the data correctly. Analyzing how the data is being processed, however, is another topic. We are not going to spend much time on it in this presentation.
Assuming that the test-case passed, our aim from learning the data is to know for a fact what constitute the "correct data". Of course, ideally, a major part of that knowledge should be obtainable from the requirement or design document. However, there is always a possibility that a couple of details are missing from those places. Hopefully we will be able to find out about it by manually executing the test-cases and doing data inspection.
So, in this presentation I will show you how to obtain the data (specifically the SIP messages), and how to relate them to the steps in the test-case.
Finally, we compile all that knowledge and transform them into the the automation code.
That would be it.
(3) Explain the application.
The application we will be testing is basically a SIP server that is capable of making a callback to someone (let's call it partyA) whenever someone else (partyB) is available. Here is the case:
There's a person Alice, who is very excited about an important business opportunity she just identified. She wants to tell her colleague, Bob, right away, on the phone. Bob was busy doing house chores. Being a responsible and honest person, he set his presence status to BUSY. The call from Alice, obviously, wouldn't get through.
Without this application, Alice would hear the ringing tone on her phone, which after 20-30 seconds would go away. No call is made. Alice, can retry later, of course, as long as she doesn't forget to do it.
With this application, Alice would be presented with an IVR that will ask her if she wants to be called back whenever Bob is available to take calls. Let's suppose Alice says "yes" (by pressing a key on her phone to indicate the choice). For Alice, it means one less thing to carry around in her head; "call bob, call bob later today". What a relief.
Now imagine Bob has finished washing the dishes, and ready to take calls. He will announce it on the web, saying "hey guys, I can to take calls now.". Well, for now, it actually will be as simple as picking AVAILABLE from the list statuses in a webpage that comes with the application, and specifying his SIP URI. [show screenshot].
At the very moment Bob makes that change, the application will establish a call with Alice, play an announcement to her ("Alice, we're going to connect you with Bob now. Please stay in line"), establish another call with Bob, and finally bridge the two calls, resulting in Alice and Bob talking to each other.
This application is a good vehicle for this presentations, because it has the following characteristics I'd like to show that we can write the automation for, using the tool I will present at the end of this presentation:
It involves interaction with IVR, and there's a need to pass DTMF signal from a caller to the application. This is quite common in telephony applications, and we would like to know if the test tool we will be using allows us to do that, easily.
It involves interaction using SIP (on the phone) as well as HTTP (using the web-browser). Furthermore, those two are interlinked. By that I mean an action performed during the call session (on the phone), will affect the state the application, and _that_ can be observed from the other side (the web browser). Vice versa. We _will_ see that the test tool allows us to automate the interaction from both sides.
(4) Explain a simple (basic / normal) test case ...
(5) Guide the audience through executing the test case manually ... Also explain how to gather the data we need, and relate them to the steps in the test case. ...
(6) Guide the audience through crafting the automation ...
