Teaser: Bare-knuckle SOA

I’m working on this idea, and I don’t know if it appeals to you guys. I’d like your input on whether this is something to explore further.

Here’s the deal: I’ve encountered teams who, when working with SOA technologies have been dragged into the mud by the sheer complexity of their tools. I’ve only seen this in Java, but I’ve heard from some C# developers that they recognize the phenomenon there as well. I’d like to explore an alternative approach.

This approach requires more hard work than adding a WSDL (web service definition language. Hocus pocus) file to your project and automatically generating stuff. But it comes with added understanding and increased testability. In the end, I’ve experienced that this has made me able to complete my tasks quicker, despite the extra manual labor.

The purpose of this blog post (and if you like it, it’s expansions) is to explore a more bare-bones approach to SOA in general and to web services specifically. I’m illustrating these principles by using a concrete example: Let users be notified when their currency drops below a threshold relative to the US dollar. In order to make the service technologically interesting, I will be using the IP address of the subscriber to determine their currency.

step 1: create your active services by mocking external interactions

Mocking the activity of your own services can help you construct the interfaces that define your interaction with external services.

Teaser:

[sourcecode language=’java’]public class CurrencyPublisherTest {

private SubscriptionRepository subscriptionRepository = mock(SubscriptionRepository.class);
private EmailService emailService = mock(EmailService.class);
private CurrencyPublisher publisher = new CurrencyPublisher();
private CurrencyService currencyService = mock(CurrencyService.class);
private GeolocationService geolocationService = mock(GeolocationService.class);

@Test
public void shouldPublishCurrency() throws Exception {
Subscription subscription = TestDataFactory.randomSubscription();
String location = TestDataFactory.randomCountry();
String currency = TestDataFactory.randomCurrency();
double exchangeRate = subscription.getLowLimit() * 0.9;

when(subscriptionRepository.findPendingSubscriptions()).thenReturn(Arrays.asList(subscription));

when(geolocationService.getCountryByIp(subscription.getIpAddress())).thenReturn(location);

when(currencyService.getCurrency(location)).thenReturn(currency);
when(currencyService.getExchangeRateFromUSD(currency)).thenReturn(exchangeRate);

publisher.runPeriodically();

verify(emailService).publishCurrencyAlert(subscription, currency, exchangeRate);
}

@Before
public void setupPublisher() {
publisher.setSubscriptionRepository(subscriptionRepository);
publisher.setGeolocationService(geolocationService);
publisher.setCurrencyService(currencyService);
publisher.setEmailService(emailService);
}
}
[/sourcecode]

 

Spoiler: I’ve recently started using random test data generation for my tests with great effect.

The Publisher has a number of Services that it uses. Let us focus on one service for now: The GeoLocationService.

 

step 2: create a test and a stub for each service – starting with geolocationservice

The top level test shows what we need from each external service. Informed by this and reading (yeah!) the WSDL for a service, we can test drive a stub for a service. In this example, we actually run the test using HTTP by starting Jetty embedded inside the test.

Teaser:

[sourcecode language=’java’]public class GeolocationServiceStubHttpTest {

@Test
public void shouldAnswerCountry() throws Exception {
GeolocationServiceStub stub = new GeolocationServiceStub();
stub.addLocation(“80.203.105.247”, “Norway”);

Server server = new Server(0);
ServletContextHandler context = new ServletContextHandler();
context.addServlet(new ServletHolder(stub), “/GeoService”);
server.setHandler(context);
server.start();

String url = “http://localhost:” + server.getConnectors()[0].getLocalPort();

GeolocationService wsClient = new GeolocationServiceWsClient(url + “/GeoService”);
String location = wsClient.getCountryByIp(“80.203.105.247”);

assertThat(location).isEqualTo(“Norway”);
}
}[/sourcecode]

validate and create the xml payload

This is the first “bare-knuckled” bit. Here, I create the XML payload without using a framework (the groovy “$”-syntax is courtesy of the JOOX library, a thin wrapper on top of the built-in JAXP classes):

I add the XSD (more hocus pocus) for the actual service to the project and code to validate the message. Then I start building the XML payload by following the validation errors.

Teaser:

[sourcecode language=’java’]public class GeolocationServiceWsClient implements GeolocationService {

private Validator validator;
private UrlSoapEndpoint endpoint;

public GeolocationServiceWsClient(String url) throws Exception {
this.endpoint = new UrlSoapEndpoint(url);
validator = createValidator();
}

@Override
public String getCountryByIp(String ipAddress) throws Exception {
Element request = createGeoIpRequest(ipAddress);
Document soapRequest = createSoapEnvelope(request);
validateXml(soapRequest);
Document soapResponse = endpoint.postRequest(getSOAPAction(), soapRequest);
validateXml(soapResponse);
return parseGeoIpResponse(soapResponse);
}

private void validateXml(Document soapMessage) throws Exception {
validator.validate(toXmlSource(soapMessage));
}

protected Validator createValidator() throws SAXException {
SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
Schema schema = schemaFactory.newSchema(new Source[] {
new StreamSource(getClass().getResource(“/geoipservice.xsd”).toExternalForm()),
new StreamSource(getClass().getResource(“/soap.xsd”).toExternalForm()),
});
return schema.newValidator();
}

private Document createSoapEnvelope(Element request) throws Exception {
return $(“S:Envelope”,
$(“S:Body”, request)).document();
}

private Element createGeoIpRequest(String ipAddress) throws Exception {
return $(“wsx:GetGeoIP”, $(“wsx:IPAddress”, ipAddress)).get(0);
}

private String parseGeoIpResponse(Element response) {
// TODO
return null;
}

private Source toXmlSource(Document document) throws Exception {
return new StreamSource(new StringReader($(document).toString()));
}
}[/sourcecode]

 

In this example, I get a little help (and a little pain) from the JOOX library for XML manipulation in Java. As XML libaries for Java are insane, I’m giving up with the checked exceptions, too.

Spoiler: I’m generally very unhappy with the handling of namespaces, validation, XPath and checked exceptions in all XML libraries that I’ve found so far. So I’m thinking about creating my own.

Of course, you can use the same approach with classes that are automatically generated from the XSD, but I’m not convinced that it really would help much.

stream the xml over http

Java’s built in HttpURLConnection is a clunky, but serviceable way to get the XML to the server (As long as you’re not doing advanced HTTP authentication).

Teaser:

[sourcecode language=’java’]public class UrlSoapEndpoint {

private final String url;

public UrlSoapEndpoint(String url) {
this.url = url;
}

public Document postRequest(String soapAction, Document soapRequest) throws Exception {
URL httpUrl = new URL(url);
HttpURLConnection connection = (HttpURLConnection) httpUrl.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.addRequestProperty(“SOAPAction”, soapAction);
connection.addRequestProperty(“Content-Type”, “text/xml”);
$(soapRequest).write(connection.getOutputStream());

int responseCode = connection.getResponseCode();
if (responseCode != 200) {
throw new RuntimeException(“Something went terribly wrong: ” + connection.getResponseMessage());
}
return $(connection.getInputStream()).document();
}
}[/sourcecode]

poiler: This code should be expanded with logging and error handling and the validation should be moved into a decorator. By taking control of the HTTP handling, we can solve most of what people buy an ESB to solve.

create the stub and parse the xml

The stub uses xpath to find the location in the request. It generates the response in much the same way as the ws client generated the request (not shown).

[sourcecode language=’java’]public class GeolocationServiceStub extends HttpServlet {

private Map<String,String> locations = new HashMap<String, String>();

public void addLocation(String ipAddress, String country) {
locations.put(ipAddress, country);
}

@Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
try {
String ipAddress = $(req.getReader()).xpath(“/Envelope/Body/GetGeoIP/IPAddress”).text();
String location = locations.get(ipAddress);
createResponse(location).write(resp.getOutputStream());
} catch (Exception e) {
throw new RuntimeException(“Exception at server ” + e);
}
}
}[/sourcecode]

 

Spoiler: The stubs can be expanded to have a web page that lets me test my system without real integration to any external service.

validate and parse the response

The ws client can now validate that the response from the stub complies with the XSD and parse the response. Again, this done using XPath. I’m not showing the code, as it’s just more of the same.

the real thing!

The code now verifies that the XML payload conforms to the XSD. This means that the ws client should now be usable with the real thing. Let’s write a separate test to check it:

 

[sourcecode language=’java’]public class GeolocationServiceLiveTest {

@Test
public void shouldFindLocation() throws Exception {
GeolocationService wsClient = new GeolocationServiceWsClient(“http://www.webservicex.net/geoipservice.asmx”);
assertThat(wsClient.getCountryByIp(“80.203.105.247”)).isEqualTo(“Norway”);
}

}[/sourcecode]

 

Yay! It works! Actually, it failed the first time I tried it, as I didn’t have the correct country name for the IP address that I tested with.

This sort of point-to-point integration test is slower and less robust than my other unit tests. However, I don’t find make too big of a deal out of that fact. I filter the test from my Infinitest config and I don’t care much beyond that.

fleshing out all the services

The SubscriptionRepository, CurrencyService and EmailService need to be fleshed out in the same way as the GeolocationService. However, since we know that we only need very specific interaction with each of these services, we don’t need to worry about everything that could possibly be sent or received as part of the SOAP services. As long as we can do the job that the business logic (CurrencyPublisher) needs, we’re good to go!

 

demonstration and value chain testing

If we create web UI for the stubs, we can now demonstrate the whole value chain of this service to our customers. In my SOA projects, some of the services we depend on will only come online late in the project. In this case, we can use our stubs to show that our service works.

Spoiler: As I get tired of verifying that the manual value chain test works, I may end up creating a test that uses WebDriver to set up the stubs and verify that the test ran okay, just like I would in the manual test.

 

Taking the gloves off when fighting in an soa arena

In this article, I’ve showed and hinted at more than half a dozen techniques to work with tests, http, xml and validation that don’t involve frameworks, ESBs or code generation. The approach gives the programmer 100% control over their place in the SOA ecosystem. Each of the areas have a lot more depth to explore. Let me know if you’d like to see it be explored.

Oh, and I’d also like ideas for better web services to use, as the Geolocated currency email is pretty hokey.