Bastard (Big Automated Scenario-Test Active Runtime-Director)

UPDATE: The source code of bastard lib is now available http://www.box.net/shared/q0f90dmmx4

Anything goes just to have a nasty-sounding product name, eh? :D

Ok, so this entry is highly related to the previous entry where I post a documentation that explains (mostly) the format of the XML file in which you describe your test scenario. In this entry I'm posting the Java library I wrote that consumes the XML and runs the scenario. Download it from here: http://www.box.net/shared/nfns0u79k9 .

Well, it's not the library per se. Instead it's a little test project that uses the library. The library itself is included in that project (look for a file named bastard_api.jar in that ZIP; that's the library you would import into your project).

Alright, time for a quick explanation. Basically you will have to create a little program -- which I refer to as "Director" -- in order to use the Bastard API. It is typically very simple. Basically what it does are: (1) Loads the content of the XML file (whose name is passed in as an argument to the program execution). Then you (2) feed that XML string into a method named loadScenarioDef that belongs to the library class named ScenarioRunner. In return you will get an instance of ScenarioDefinition. Further you (3) create an instance of ScenarioRunner and spawn a thread to run it. There's an instance method named run that belongs to ScenarioRunner, into which you will have to pass the scenario definition you created in step 1 and the instance of mediator (see the doc in the previous entry to understand the meaning of it) .

Some screenshots...:

Step (1)

Step (2)

Step (3)

As you see in the screenshot of step 1, you will have to pass the path to the scenario XML file as the argument to the program execution. The project comes with several XML files that you can use to play with. Try duh.xml first (for its nice name). If you’re using Netbeans, use the following screenshot as a reference.

By running FrijolDirector with duh.xml, you should see this in the output console:

Okay. That’s all for today. Feel free to mail me in case you have trouble running / trying / using it. Saludos.

Automated Scenario-Test Descriptor

Hi,

It's been a long time, can't do much right now, lottachanges, will write about that later. However, I feel the need to share this document I wrote a little while ago; it's a documentation of a test-tool I wrote for the work.

The context of this post: Cataroo

Well, through the process, I learned two things, I believe:

• I kind of see the value of functional programming now.
• I feel the need to learn Scala, for its promise of better support for conccurency / parallelism.
• I see the value of Continuation. I had to keep the number of active threads below certain limit. There are situations where a thread have to wait, but I can't simply use sleep(), because it would't release the thread. So, in Java, without Continuation, I had to do some tricks to achieve the effect pausing / resuming thread (basically I had to make the thread stores its current state somewhere, before entering "paused mode", and immediately exits its run() method. Later, when the thread has to be resumed, another process will retrieve its current state, kick it to run, and pick up from where it left off. Tricky, could've been easier with Continuation, I guess).

Feel free to ask me if you want to know more how it was designed and implemented.

UPDATE: The source code is available at http://www.box.net/shared/q0f90dmmx4

UPDATE: the JAR of the library can be downloaded from http://jananuraga.blogspot.com/2010/01/anything-goes-just-to-have-nasty.html ... the PDF can be downloaded directly from Box.com: https://www.box.com/s/o9jrdseo12u1oieemnxa
Automated Scenario Test

Another random stuff; randomness continued....

[This is a continuation of this entry]

I just finished (quick)reading a book titled "Charging for Mobile All-IP Telecommunications"..., and I think it's a good book. It helps me get a little more into the detail of diameter (protocol). Well, this book does not only explain diameter / charging in IMS network; it also has chapters on charging in GSM and GPRS network. But the part that interests me more of course are the parts on IMS. What I like about the book: it explains the message-exchange for each charging scenario, step-by-step, with some concrete example values to further illustrate the point(s). It helps a lot.



Well, that's all my review :). Ok, I'm not so good at it. Anyway, this charging stuffs is a soup of acronyms (yuck). Here are the acronyms I think I need to know / care about for this moment: CDF, OCS, Ro, Rf, and HSS.

• CDF (charging data function): the element that receives charging events from various other elements in the (IMS) network, and constructs the complete CDR, before (eventually) handling it over to the billing system.

• HSS (home subscriber service): the element that stores user's profile (authentication info and subscription profile). The authentication server queries this element (that goes without saying).

• OCS (online-charging system): the element that calculates service-usage charges. It is primarily used in prepaid-charging scenarios. So, whenenever a service is to be consumed by a subscriber, the SIP proxy / AS sends a request to the OCS, that will trigger a price-calculation on the OCS. Assuming the subscriber still has credits left in her account, the OCS will return a success response, containing the amount of time / volume granted, and the service starts being delivered. In case the service is session-based, the SIP proxy / AS will periodically send other requests, similar to one sent at the beginning, asking for additional credit-usage, to the OCS. All those message-exchanges go through the Ro reference point.

• Ro: the "reference point" between the CDF and SIP proxies (and the SIP application server) in the (IMS) network. It is used in offline charging. The way I understand "reference point": diameter defines various reference points. I visualize a reference point as a line that connects two elements in the network, and it defines its own set of messages that can be / should be exchanged between those two elements, and in what order. Well, if diameter is a protocol, then I'd say "reference point" is a protocol inside the protocol..., or sub-protocol. So, for each event / session each it manages, those SIP proxies and application server (AS) sends a message to the CDF. Using some correlation mechanism, the CDF builds the complete CDR based on the messages it receives from those proxies and AS.

• Rf: the "reference point" between the OCS and the SIP proxies and SIP AS.
  

Ok, those are the factoids. How am I going to use them? Well, I was trying to estimate the development effort for the teleconference system, especially the parts related to charging. First, I need to know what needs to be built. Let's go over them again:

• CDF: I think we need to develop it. But I guess the "correlation function" would be quite tricky to implement; making it conform to a standard / RFC needs bunch of love, I mean, verifications and validations. I was wondering if there's any open source project, that can be used as a reference / example / base / building block. So far I haven't seen any. Anyway, if we have to build it we need a way to communicate with the SIP proxies / AS (to receive diameter requests, process them, and send the responses), and libraries like jdiameter (or openblox) can be very useful (we'll be using the diameter server library, particularly the one that abstracts away the Rf reference point).

• HSS: is there such thing as "standard of structure of user profile, subscription profile, etc" for HSS? I guess there is none. At the end, it's "just another database application" in my opinion. You have more freedom in the implementation of your HSS, as long as it can be accessed through Sh reference point. Hmm.... On a second thought, implementing HSS should not be our priority. There are two possible situations: (a) the buyer does not have HSS already, or (b) the buyer already has HSS (e.g.: a telco operator that owns an IMS-network).
  
  In case (b) we'd better spend our effort more on making sure our teleconferencing system will be able to exploit the richness in the structure of the operator's user / subscription profile (which reflects the operator's charging requirements)..., and vice versa: making sure the operator can exploit the exploit the richness of "chargeabilities" (and functionalities) of our teleconferencing system. In short, we'll have to build an adaptor, a two-way adaptor I'd call it. My tactic would be: (1) brainstorm and investigate the possible ways of charging the users of our teleconferencing system, (2) obtain all the information we can have regarding potential buyer's charging requirements / capabilities, (3) prioritize the list we made in step 1, taking into account the information we obtained in step 2.
  
  Hmm..., allright, what about case (a)? My short answer for this moment: implement our own HSS. I mean, we define our own schema for storing user and subscriptions profile (just a database application, really, in my view), and provide a thin wrapper around it such that it can be queried using diameter protocol (through Sh reference point). The design of that schema would go in lockstep with the design of the functionalities (thus chargeabilities).

• Back to CDF: I think the situation is quite similar to that of HSS: (a) the buyer does not have CDF, or (b) the buyer already has CDF. Frankly, I have a couple of doubts related to CDF. First: so, this CDF is the one responsible for the construction of the complete CDR out of the events received from various nodes, by executing the correlation function. But who defines the format of the CDR? Is it tied to the billing system that is used? I guess the answer is no, because there's a mediation element (part of the billing solution) that performs the translation / mapping of the CDRs it receives (in any format) to a format suitable for that billing system. So..., who? Some standard body? I tried googling for it, including looking at the Parlay specs., but nada. Is it the operator (that owns the entire system)? So, in that scenario, the service provider will have to deliver CDRs according to the format specified by the operator. Hmm..., I guess the answer is no, for the same reason: that's why they have the mediation element. What is left then: the service provider itself, us (with close collaboration with the operator, supposedly).
  
  For the sake of discussion let's assume the latter is true. In that case, the CDF will produce CDR in various formats (one for each service). To me that implies: there are several "correlation functions" installed in that CDF. I picture this "correlation function" as a plugable module (one for each service). Who then is responsible for the implementation of the correlation function of a service? I guess the answer is the service provider itself (again, with close collaboration with the operator), because it posses a better knowledge of the events generated by the service (and their correlations).
  
  Eh, wait a second..., am I wrong to think that CDF overlaps with meditation in several areas / tasks? But, hey, there's another element between the CDF and the billing system; the "charging gateway function" (CDF)..., and to me (again) it overlaps with billing mediation. This starts to confuse me.
  
  This whole questioning got me thinking: wouldn't it be easier (and better) if we get a copy of a charging platform that has CDF and HSS, evaluate it, see how it works, see how it can / should be customized or extended in order to make it work with our teleconferencing system. That strategy will help us cast our doubts more quickly. After all, we're not in the business of charging platform (eventhough an intimate knowledge of such platform will be very helpful in designing and selling our solution).
  
  So I googled around, and here are what I found:

1. http://www.digitalroute.com/product/productsolutions/ims/
2. http://www.sitronicsts.com/en/products_and_solutions/telecoms_and_media/bss_oss_product_lines/foris_charging_and_billing.html
3. http://www.intecbilling.com/Intec/Products+Services/Products/Charging+and+Billing/Intec+IMS+Charging+Solution.htm
4. http://www.openet-telecom.com/
5. http://advoss.com/IMS_offlinecharging.htmlhttp://www.marben-products.com/diameter/overview.html
6. http://www.tieto.com/default.asp?path=1%3B93%3B16080%3B124%3B16837%3B17193%3B16242%3B36978%3B37078
   
   I had to check the whole set of returned links by google queries: +IMS +charging and "charging data function" just to get that list :).
   
   Allright..., next what? Hmm.... well, I guess I'll just leave it here for now. On to OCS.

• OCS: Well, like I already stated, if the 3rd-party charging system we use already has it, then fine, not much to say here. But if not, no other choice we'll have to build it in-house. It is a component that resides very close to the billing system (where subscriber's account information is managed), so I think the OCS can be made as an extension to or something that is built on-top of the billing system. I don't have much time right now to do some more research on this, but if you'd like to know some more, here is an interesting, relevant wikipage: http://en.wikipedia.org/wiki/Online_charging_system.
  

Finally, I'd like to turn our attention to the application server, where the core of our conferencing server, supposedly, will be deployed. The application will be sending events to various charging elements. It is quite likely that it would be something that changes quite often (i.e.: we add / change feature, we change the structure of service and user profile, etc..., and somehow it would impact the chargeability of the application, hence the the events need to be sent). I was thinking: how to design and implement the application such that those changes in charging would not imply direct change in the main code (whose concern is mainly the administration of live conferences and activities within the conference room). Could it be that aspect-oriented programming is the right answer?

Lastly, for those who like to see things explained with lines of code (me too), here's a nice article from Oracle: "Understanding IMS Charging Architecture"..., and the use of diameter API in BEA (oracle) Weblogic SIP server is detailed here:

• Using the Diameter Rf Interface Application for Offline Charging.
  
• Using the Diameter Ro Interface Application for Online Charging.
  

One last note: I don't mean to suggest that this conference solution will be designed specifically for deployment in IMS network (only) with this whole rambling about charging-related IMS elements. We should take it at the conceptual level. I mean, at least we should know what things we need to be considered as far as charging is concerned: there should be something that stores user and service profile (HSS), there should be something that correlates the charge-inducing / charging-related events sent by the conference server (CDF), and there should be something that calculates the charge, on-line (OCS). That's all. The names don't really matter. However, by following / learning from how those elements are organized in an IMS network, it will be easier for us to deploy our solution in an IMS network (when the need / opportunity arises). Beside that organization is pretty common I believe, and makes a good sense.

Alright, another last note :): (and this is more a reminder to myself): this whole desk-research over charging elements should not distract us from our main objective that is "to make our conferencing solution (more) appealing, also from charging and billing perspective". I think one way to achieve is by making sure (in the design) that our solution allows us / whoever buy this solution to be as creative / innovative as posssible in charging the users :).

Yet another segment of chapter 4

Another little chunk, still in chapter 4, on implementing click-to-dial application using SIP Servlet. In this chunk the use of Re-INVITE message is shown and explained for the first time.

As usual, we scrutinize-and-map the SIP-flow diagram before doing any actual coding. I hope you like this one, as well ;).

[Context: video series]

Random stuffs #1

Tired.

I feel like crawling to finish the chapter 4 of my video series. Well, to be perfectly honest I haven't done anything about it in the last two weeks. Sigh. I feel exhausted. I need gado-gado, sate babi, sate lilit, tum ayam, ayam betutu, nasi padang, nasi kuning, lawar bali, nasi tim ayam, nasi uduk, etc. They're all my fave indonesian foods :). Ok, one more saturday for chapter 4, and that's it. I will have to use a different strategy for chapter 5 (the final chapter) that talks about VoiceXML. I probably will record directly the training session on VoiceXML I'll try to set-up in the new company I'm working for. That will save time.

Anyway, news: I just changed workplace. I have done my service as system analyst (on the side of nextel mexico) in the implementation of their new rating and billing system (using a product named BSCS iX from a company named LHS). It was a 11-month assignment and it served me well, I got to know the possible parameters & scenarios for charging and reference about a commercial system for that purpose among other things.

Now, still in Mexico City, I work for a startup company that specializes in development of VoIP system, and more specifically teleconferencing.

In this entry I'd like to sum up a couple of things I've been thinking the last couple of days. First: I want to unify the knowledge I've gathered so far in rating & billing, SIP servlet, and VoIP in general in my effort of designing a complete teleconferencing solution (and business). Here's the list of technologies I'm currently looking at:

(1) Conferencing bridge

We have things like Asterisk (with MeetMe or ConfBridge), jVoiceBridge, and some other commercial solutions (e.g.: Avaya's spectel). I know a little about how to use MeetMe in Asterisk, I played around a bit with it in 2006.

jVoiceBridge provides a good documentation at architectural level that helps me get a better idea of the functions / possible uses of a conferencing bridge. Additionaly, someone wrote a nice intro to jVoiceBridge.

The design of the conferencing solution, of course, will have to accomodate the need to interface with several conferencing-bridge products. So, we'll have to build an abstraction layer. But, still, I need to know how those choices compare against each other in term of feature-set and performance, in order to be able to define a lowest-common denominator and prioritize the development effort.

(2) Rating-and-billing (including online-charging, which requires the use of AAA protocol).

This is important because... well, it goes hand-in-hand with the business-model. I mean, the business-model dictates the requirement for the rating and billing system. This is a top-down view in my opinion.

On the other hand, the more well-designed (flexible, extensible) the rating-and-billing system is, the more room is there for the business model to grow / move. This is the bottom-up view.

I'm wearing the engineer hat at this moment; I'm taking the bottom-up view. I need to know what level of flexibility in rating-and-billing is possible with current (readily available) tools; how far we can go. So..., I tried getting opensips + freeradius + cdrtool installed. The reasons: (a) that's the first one that appear on google for my query :), and (b) I've heard good comments about opensips, especially when it comes to performance.

I got opensips installed without problem on my debian linux. But it's completely a different story with CDRTool. The installation guide is not complete (a couple important stuffs are skipped. Apparently it assumes reader's familiarity with integration of freeradius with opensips). At the end, I failed to get it working. I wish somebody had written a better, more thorough, well-written and well-tested manual. Please let me know. I decided to drop it for a while; it already took my entire sunday last weekend!

(3) Still in rating-and-billing.

To my understanding, the combination of opensips + freeradius + cdrtool provides us only a way to decide whether a call is allowed (by querying freeradius), collect the SIP trace, store them in the database, correlate the INVITE and the BYE of a SIP session, calculate the duration, and apply the rating formulas to the session.

Another component is needed for enabling online charging, that allows the call to be terminated once the credit reaches certain threshold. AG-Projects (the same company that produces CDRTool) has a product named "Call Control" just for that. It works together with CDRTool (as the rating engine) and opensips module named callcontrol that queries the "Call Control" application every predetermined amount of time to find out whether or not the call should be dropped. That's how I see it.

However, I just get the impression that CDRTool + CallControl only address the basics of rating. In order to make it commercially viable, it has to be augmented with other software packages that allow us to work with concepts like rate plan, rating package, promotion, free units, hierarchical account, etc. Please note I'm using my knowledge of BSCS iX as a reference.

I came across this one JBilling, an open-source billing software. From a quick glance, it seems to be feature-full enough. My upcoming tasks would be: buy the manuals of jbilling which in total would cost around 140 USD (or 40 USD without the extension and integration guide), evaluate it, and later investigate if it makes sense & worth the effort integrating it with opensips + cdrtool + freeradius + callcontrol.

Worth the effort? Well..., I was a bit put off by the difficulties in getting opensips + freeradius + cdrtool to work, so I started to think: "If it's only about collecting SIP traces, detecting the start and stop of a call, and applying some simple rating functions then what's the big fuss about?". Allright, rating might be the not-so-easy part, but -- assuming jbilling is the right thing -- rating engine is already part of jbilling (see the general architecture diagram of jbilling).

So, what is missing are CDR collection, online-charging (ability to disconnect the call once the credit is up), and authentication-and-authorization. Doing all of them in-house doesn't sound unreasonable to me. I think we can use SIP-servlet to implement them, and deploy them together with jbilling as a single deployment unit.

(4) Newer AAA (authentication, authorization and accounting) protocol: diameter

My last concern about opensips + freeradius is the protocol it uses: radius. Well, there's a newer protocol, diameter. What's the real advantage it offers to the conferencing system I want to build? I don't know yet, but if we plan to sell this solution to telco operator that uses IMS then I believe we have to work with diameter (as diameter is the AAA protocol of choice in the IMS architecture).

So, again, that is another reason not to use opensips + freeradius + cdrtool combo in our solution, but instead implement the interfacing with diameter-based AAA system on our own. Furthermore, I have to think about providing a layer of abstraction in our code that would allow us to sell this solution to two kinds of customer: (a) those who don't have rating-and-billing system, and (b) those who do. For customers of type "a" we can bundle it together with our rating-and-billing system (based on jbilling, for example).

Ok, again, assuming this part of the conferencing solution will be implemented as an enterprise java application containing SIP servlet components, then the availability of diameter stack in the SIP servlet container will be very much welcome. Fortunately that's how it is now: Sailfin will (or already?) have diameter stack (using OpenBlox), read it here: http://blogs.sun.com/theaquarium/entry/sailfin_diameter_support_online_charging. The same thing with Mobicents from Redhat, it comes with diameter stack from jdiameter. More detail here: http://groups.google.com/group/mobicents-public/web/mobicents-diameter.

[Update: continued on this entry]

First stab at chapter 1

Where do I start? Allright, this picture.

Now the context: last monday we met in my place in mexico city. By we I mean: me, Eric, and Antonio. For the recording of chapter one of the video series. There was an interesting talk (platica), mostly between Eric and Antonio.

Ok, this should've been in the video, but we simply didn't / forgot to do it :) No introductions recorded. Therefore allow me to put a brief one here.

Antonio Echeverria works as a solution architect in one of those vendors that are pushing IMS. We knew each other when I was working in a project for an (fixed) operator in mexico. His role / job in the company allows him to get some insights of the current situation of the IMS, business and technical-wise, especially for the mexican market. He's been a telco domain for quite a long time, and I'm fortunate to be able to learn things from him.

Eric Werkhoven was my colleague in the company we used to work for during 2005-2006. The company was developing a hosted call-center solution. Currently he works for another VoIP company producing & providing outbound / predictive dialing solution. A deep understanding and appreciation of the technical challenges in developing and deploying of large-scale VoIP solution is something that I learn from him now and then.

Antonio and Eric are from two different worlds that are colliding (or converging?). I was happy to be able to get them together and talk, exchanging their views, which might not be completely aligned to each other.

The talk we had last Sunday was mostly about the business-models / opportunities around IMS. We think that's the most important point to grasp before delving into technical discussion. I personally am interested to know the value-chain; to see where I -- viewing myself as a (potential :D) service provider -- might be able to fit in, and how to profit from it. That's what I would like to share with others in the video(s). Of course we don't pretend to become an authoritative source for that kind of information. We just wanted to capture our learning process. Here's the unedited conversation I managed to record (only 38 minutes).

Unfortunately, we only had an hour to get together on that day. Because of my stupidity I also forgot to record the first 15 minutes of the conversation (when we were discussing the flow of the dialog); some interesting points were brought up there.

So here I am now, grappling to collect those points and make up for the lacks. I'm trying to back up some of those points with some references I can find on the internet / other sources. Hopefully this can improve the value / usefulness of the information I'm going to provide in the video.

Ok, so that's the context. I promise to post those points as a written text in a couple of days, before presenting them as a video. The point I'm presenting here -- just one point -- is a prelude to that post.

Content proxy. That's one thing that came up during the discussion. We were exploring the scenario where the operator acts as a proxy to the contents in the internet. In Eric's opinion, telco operators are not capable to compete with the internet. The rate of innovation that's happening in the internet is unparalleled. So the operator is not supposed to go down the path trying to (re)create the existing successful services / content on the internet. Basically he considers the thinking that "with IMS the telco operator can be like Google" is misguided at best.

So, being a content proxy might be their (only?) option. Let's go back to the diagram. First and the foremost, IMS is supposed to provide the same set of basic telephony services that people already enjoyed with the current, non-IP, technologies. That's what line A represents.

IMS lives in an IP network (after all IMS is an abbreviation for IP multimedia subsystem :P). In order to be able to fully deliver its promise(s), the clients also need to be in the IP network and have SIP-stack installed (among other things that make them IMS-capable). There aren't enough mobile devices in the market with such characteristics. In fact, it is an obstacle to a wide-spread adoption of IMS.

However, people can already enjoy high-speed access to IP network; 3G. Mobile devices have improved significantly. Things like iPhone, for example, turn lack of screen-space and crippled web browsers into things of the past. People can have direct, unabridged access to services like YouTube, Last.FM, etc. That's what line B represents.

The "content proxy" idea, to my understanding, is like trying to drive the subscribers away from line B to line C. People will do that willingly if they perceive additional values. What kind of additional values can be provided (with access through line C)? What does IMS -- and only IMS -- have to offer that enable such services? There was a relevant point raised during our discussion (that might provide an answer to those questions). It has to do with subscribers' profile (stored in the HSS) or rating & billing system (that is a complex stuff to install, configure, and manage correctly). Operators might be able to bank on it in their IMS offerings. That idea is still not perfectly clear to me, give me time to investigate it further.

In the meantime, please chip in, help me enrich the discussion, help me arrive to the right conclusion.

Thanks in advance and best regards,
Raka

UPDATE: When the recording started we were talking about Connected Home that Antonio mentioned to us. It got me thinking of the effort from Ericsson to push further the widespread of this telco 2.0 app. development, by providing hosting for SIP-servlet applications (on Sailfin), and some helper APIs and other stuffs. Check it out on the Ericsson Labs page.