Preparation for Live Mobile Hacking with an OFM 12c red stack
On March 4th, I am to present – together with ADF and Mobile Application Framework expert Luc Bors – a live development demo session at the EMEA Oracle Fusion Middleware Partner Forum in Budapest, Hungary. Luc and I are in the middle of our preparations for this event. And I thought perhaps it would be nice to share some of the background for this session.
It all started in the Fall as Jürgen Kress, responsible at Oracle for Fusion Middleware Partner Adoption EMEA, sent out a call for papers for the Forum, looking for proposals for presentations and other types of sessions. Luc and I prepared a proposal for a session where we would do live development (always appealing for the audience and somewhat nerve racking for the presenters) and show the development of a mobile app (using Oracle MAF) on top of a mobile back end (created using SOA Suite 12c and its REST capabilities). Luc and I have done similar sessions in the past. They can be a lot of fun – and be quite stressful because of all the things that can and typically will go wrong.
Jürgen accepted our proposal, invented the title (Live Mobile Hacking with an OFM 12c red stack) and allocated a general slot for us: a full hour as the last section of the second day.
That was several months ago, and now the event is imminent and we better start preparing in anger. First of all we needed a case: what would be the storyline for our live demo? We decided on the airline industry – inspired perhaps a little bit by the airport case that provides the backbone for the book I am currently writing on SOA Suite 12c. A flight provides a good example of an application that does not have constant connectivity – one of the features we would like to demonstrate. Everyone partakes in flights from time to time. And between flight attendants, ground staff and the back office systems of the airline and the airport, it would not be hard to find interactions that we could use in the demo.
Last week we started discussing the case in anger. And we ended up with the following storyline:
The mobile app runs on a tablet and is to be used by flight attendants. The app allows the flight attendant to retrieve details about today’s flight. Then, the passenger list can be presented and the status of each passenger can be verified and updated. The status of the flight can be updated, from waiting to boarding and from boarding to closed. These updates are immediately sent to the back office for synchronization to the terminal displays. During the flight, with the app in offline mode, the app can be used to record complaints and suggestions from passengers, for example with regard to malfunctioning entertainment systems, immovable chairs, great or poor food and rude behavior by the cabin staff. Details are recorded including passenger contact information. When the plane has landed, the app reconnects and the data recorded in off line mode is uploaded to the back office. For each complaint, a BPM process is initiated and an email is sent to the passenger who lodged the complaint. A human task (or even an ACM case?) is created to handle whatever the complaint is about.
The Outline of the Session
Once the case was decided, we could create the outline for the session itself. What is a good way to bring across the main concepts? What will fit in one hour? What is the logical order to explain and demonstrate? What are the essential elements that we should include and create live – and what should be prepared and only demonstrated in the end-to-end case? We quickly decided on the following outline (still open to change):
10 Intro of the case and the technology stack; overview of the end to end case/application architecture
20 Brief intro SOA Suite 12c; Intro/demo existing SOAP services; demo REST/JSON enabling of SOAP services; demo of REST services from browser or SoapUI
20 Brief intro of MAF? Demo development App and UI and the interaction with REST services
5 End to end demo of a flight (flight details, passenger list, flight status (boarding, closed); record complaint about TV screen and start of BPM process (send mail)
5 Short summary of what we have shown and demonstrated, of how this type of development (Mobile App/Mobile Oriented Architecture/SOA back end) usually goes; some conclusions/tips
It should be doable, it does paint a useful and fairly complete picture. It will be challenging – so many different moving pieces, so many things that can go wrong and mundane things like switching the cable to connect to the projector between our laptops and finding a way to create the connection between our laptops.
Luc and myself hardly ever meet in person. Our communication is through email mainly. So actually collaborating through close and intense personal interaction is not really an option. The architecture for our application does not really require very frequent conversations: the layered, service based approach allows for a very decouple way of discussing, designing and implementing the application – a core strength of the approach. Luc does not require in depth knowledge of SOA or service implementation and database interaction and I need no knowledge at all about the Mobile Application Framework and the development of a mobile UI and its features.
Crucial however is the careful design of the interfaces of the services and their operations – or in REST terms: the resources and their attributes. What resources are available (Flight, Passenger, Complaint) and which operations are supported on each resource? What search parameters are supported? What is the (JSON message) format for the request and response messages? What are the (URL) paths for the resources?
Once Luc and I have agreement on these interfaces – we can do our work in splendid isolation: Luc can work from a mock implementation of service interfaces upwards to create the mobile application and its capabilities. Meanwhile, I will create the back office systems and services: the corporate database for the airline, SOAP Business Services and the BPM Complaint process. Then I will use the REST adapters in SOA Suite 12c to expose the agreed upon service interfaces as both RESTful and JSON-speaking. The business services and the RESTful endpoints can be tested and when accepted can be made available to Luc. He replaces the reference to the mock implementation of the service interface with the end points for real services and the end -to-end chain is established, all the way from the mobile app with off-line mode to the corporate database and process engine.
Designing the resources and operations has to be done carefully, as changing design details can have substantial impact. The exact definition of the attributes and the JSON message format is not as important because we are quite flexible in changing those at a later stage.
The operations my RESTful services will provide to Luc for consumption in the MAF app are:
retrieve flight details: GET – http://host:port/FlightService/FlightService/flights/<flightCode>?flightDate=<date of flight>
retrieve passengerlist for flight: GET – http://host:port/FlightService/FlightService/flights/<flightCode>/passengerlist?flightDate=<date of flight>
update status of flight: PUT – http://host:port/FlightService/FlightService/flights
file a complaint from a passenger: POST – http://host:port/FlightService/FlyingHighCustomer/soa-infra/resources/default/CustomerService!1.0/CustomerCareService/complaints
In a follow up article I will provide details about the back office (SOA Suite, BPM Suite, Database and the interaction between them and the exposure through REST/JSON of them) as well as how we made our two laptops work together.
Mobile backend with REST services and JSON payload based on SOA Suite 12c for Live Mobile Hacking with an OFM 12c red stack
For an audience of some 200 SOA Suite and Oracle Middleware experts – including the product management teams for several of the products we were going to demonstrate – we were to develop live both a mobile app as well as a mobile backend (REST APIs) to support the mobile app.
The use case was defined – an app for flight attendants to be used on tables to look up flight details, inspect the passenger list, update the flight status and record passenger complaints. Luc handled the mobile app based on the agreements we made on the REST resources, services and payloads. It was up to me to implement the backend – consisting of a database, a SOA composite to expose the database data and operations at middleware level (in SOAP/XML) and a Service Bus project to expose the required REST service interface with JSON payload, leveraging the SOAP/XML service published by the SCA Composite.
This article will briefly describe the steps I went through. All source code for the mobile backend can be downloaded from a GitHub repository.
Step One – Database
The database created for Flying High contains just five tables – as shown in the next figure.
However, these tables are encapsulated – hidden from view from external consumers. This I consider to be a best practice. For various reasons that include the encapsulation of (potentially complex) SQL statements in APIs in the database rather than in client applications and the freedom of changing the actual implementation of tables and the way data is stored. The API that is externally exposed is a PL/SQL package FLIGHT_SERVICE. The package specification contains three unit – two functions and a procedure – for retrieving flight details, the passenger list for a flight and for updating the status of a flight.
The input and output parameters for these units are all based on user defined types. This allows me to exchange nested data structures in single round trips, provide data structures that map very well to XML messages and that can very well be handled by the database adapter. Additionally, use of these UDTs allow the database developers to work in a rather elegant way – leveraging their tools to their full potential.
When you check the script database.sql in the source code you will find the definitions of the tables, the UDTs and the PL/SQL package.
Step Two – SOA Composite, Database Adapter, SOAP/XML Service
Each of the operation in the database package Flight_Service is exposed as Reference in a SOA Composite application using three database adapter bindings. The User Defined Type based inputs and outputs to these operations are represented in (very similar) XML structures in the interface definitions for these three bindings.
In parallel with the creation of the PL/SQL package, a WSDL was created for the FlightService, with a similar granularity in operations and message data structures. Note: this is not required, but it certainly makes life simpler when you have to do a pressure cooker demo. The WSDL is visualized here:
You can access the source itself through this link. The FlightService.wsdl has a companion XSD document that describes the request and response messages as we want to expose them to consumer of the SOAP/XML service.
The structures in the XML Schema Definitions are quite close to the User Defined Types in the PL/SQL API. However, they are defined from a canonical model – not from database table and column definitions.
A Mediator component was added to the
For each of six back-and-forth hand overs, I have created XSLT maps to convert from canonical input message to the input required by one of the three database adapter bindings as well as to convert from each of three outputs from the database adapters bindings to the response message format specified in the FlightService WSDL and XSD.
The Mediator has three routing rules configured that each use two XSLT Maps. The maps are quite straightforward, thanks to the similarities between the PL/SQL API object structures and the XSD definitions. One example of an XSLT map is shown here:
This map uses a DVM (domain value map) lookup to convert between the code used for FlightStatus in the database (sch, brd,cld) and the canonical values to be used in the response message (scheduled, boarding, closed). The DVM definition shown next. The document was transferred to the database MDS under the runtime SOA Suite.
After deploying the SOA composite (and configuring both the Data Source and the Database Adapter connection), the FlightService can be invoked, for example from SoapUI:
This screenshot shows a call to the getFlightDetails operation and the resulting response – wrapped in a SOAP envelope and of course all XML in nature.
Each service call will start a new instance of the FlightService SOA Composite. The instance will make the call to the database. All interaction can be tracked – with a fine grained enough audit level at least – as shown in the next figure.
This figure shows the XML structure that came out of the Database Adapter binding – and that was converted from the User Defined Type at database level. Note the value of BRD for FLIGHT_STATUS that is converted by the DVM lookup to boarding.
Unfortunately this beautiful service is still not good enough for Luc and his Mobile App. A REST/JSON service was the agreement. So my next challenge: expose the FlightService as a REST service that uses JSON payloads.
At this point, the Mobile App can come off the mock services it has undoubtedly used during development, and start consuming the real services.
Step Three – Design REST services, Resources, URLs and JSON payloads
Luc and I had discussed the functionality to be offered in the mobile app. Subsequently, Luc indicated the REST services he would require to implement the app. Both in terms of functions and in terms of the actual URLs patterns and JSON payloads. The latter can be found in this GitHub directory.
The former were defined as follows:
To retrieve flight details: GET request to /FlyingHigh/FlightService/FlightService/flights/<flight code>?flightDate=2015-03-07 (with a JSON response structured as shown below)
To retrieve the passenger list for a flight : GET request to /FlyingHigh/FlightService/FlightService/flights/<flight code>/passengerlist?flightDate=2015-03-07 (with a JSON response structured as shown below)
To update the status of a flight : PUT request to /FlyingHigh/FlightService/FlightService/flights – with a JSON request payload as follows:
The REST service for submitting a complaint from a passenger was defined the other way round: I dictated the service interface and Luc simply had to consume it – as an example of how ideally you should not do things in mobile app development. The call for submitting the complaint has to be sent as a POST request to: soa-infra/resources/default/CustomerService!1.0/CustomerCareService/complaints with a JSON payload structured like this:
Note: the CustomerCareService is exposed from a SOA Composite directly, not through the Service Bus proxy that is created in the remainder of this article. The flow for requests to this service is as follows – a BPM process being triggered as a result:
Step Four – Create REST proxy services in Service Bus project
The REST adapter wizard in JDeveloper can be used in two ways: either take an existing Pipeline (Service Bus) or Component (such as Mediator or BPEL) and derive a REST binding from the existing SOAP/XML service interface (not an ideal approach) or design a REST service with corresponding payload as desired and have the adapter wizard generate the internal SOAP/XML representation that can be connected through a Pipeline or Mediator. The latter is the approach to be used when the REST service design is leading – as it usually is, including this case.
The starting point is a Service Bus project that already contains a Business Service for the FlightService SOA Composite. This business service obviously exposes the same SOAP/XML interface exposed by the composite.
What we are after, is a REST service exposed by this Service Bus project – of which the interface definition is predefined by Luc. We can create this from the context menu in the Proxy Services swim lane. The wizard for configuring the REST binding appears.
Set the name to FlightRestService.
Now for each operation the service should expose, a Resource should be configured. For example for the retrieval of FlightDetails using /FlyingHigh/FlightService/FlightService/flights/<flight code>?flightDate=2015-03-07
Press the gears icon to specify the format for the response payload. This is expressed through an XSD document that describes the native format for the response; we call this type of document an NXSD document – in this case for example nxsd_retrieveFlightDetails.xsd.
Choose the type of format:
And load the sample of the JSON message payload:
The wizard will now create the NXSD representation of this JSON message format:
Close the wizard.
Back in the Response tab, we can generate a sample of the message payload in JSON or XML format:
The operation to fetch the passengerlist can be defined a similar way:
and the details:
Note: the endpoint definition for the REST service is set like this – a bit silly really, with the duplicate FlightService that we will have to use when invoking the service, for example from the browser:
Step Five – Create Pipeline to connect REST proxy to Business Service
Create Pipeline to connect REST proxy to Business Service for SOAP/XML services with appropriate transformations between message formats (JSON/XML and vice versa).
Add a Pipeline (for example FlightPipeline) that implements the WSDL created for the REST Proxy Service. The REST proxy can invoke this pipeline, handing it the XML representation (described by the NXSD document) of the URI parameters received in the REST call. It expects to receive the XML representation (also described in the NXSD) of the JSON payload of the response message.
XQuery of XSLT based transformations are required between the NXSD description of the request and the XSD description of the input to the business service FlightServiceBS and between the output from the FlightServiceBS and the required NXSD format to be returned to the REST Proxy Service.
The next figure lists the six transformations created between NXSD and XSD-for-Business Service:
The pipeline between REST proxy and Business Service contains an operational branch – for the three REST operations in the FlightService. In each operation, it does similar things: transform the request, route to the business service and transform the response.
The overall project looks like this:
Step Six – Publish Service Project and test end-to-end REST services
Nothing special about deployment, so I will skip that part.
After deployment, the REST services can be invoked. An example is:
This results in a SOA composite instance with the following flow trace:
We can look at the details of what happens inside the Mediator:
to learn that what we get in JSON is pretty much what was constructed in the PL/SQL package in the database. No surprise there.
Of course I can invoke the other REST services as well:
but that does not add much value.
The really interesting next step is when Luc starts implementing the mobile app against these REST services. Perhaps the Passenger Complaint is mildly interesting – as it causes some activities, as illustrated by the next two pictures.
First the call to the REST service for (POSTing) customer complaints:
and then the result inside the SOA Suite and BPM Suite:
Lucas Jellema, active in IT (and with Oracle) since 1994. Oracle ACE Director for Fusion Middleware. Consultant, trainer and instructor on diverse areas including Oracle Database (SQL & PLSQL), Service Oriented Architecture, BPM, ADF, Java in various shapes and forms and many other things. Author of the Oracle Press book: Oracle SOA Suite 11g Handbook. Frequent presenter on conferences such as JavaOne, Oracle OpenWorld, ODTUG Kaleidoscope, Devoxx and OBUG. Presenter for Oracle University Celebrity specials.