Showing posts with label rest. Show all posts
Showing posts with label rest. Show all posts

Friday, January 18, 2019

Spring REST testing

After too much Internet searching, I was unable to find an easy solution to repeated duplication in my Spring MockMVC tests of REST controller endpoints. For years now, the endpoints we write have typically sent or received JSON. This is what I mean:

mockMvc.perform(post("/some/endpoint")
        .contentType(APPLICATION_JSON_UTF8)
        .accept(APPLICATION_JSON_UTF8)
        .content(someRequestJson))
        .andExpect(status().isCreated())
        .andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_UTF8_VALUE))
        .andExpect(header().string(LOCATION, "/some/endpoint/name-or-id"))
        .andExpect(content().json(someResponseJson));

All the repeated "APPLICATION_JSON_UTF8"s, in every controller test!

If there is an existing Spring testing solution, I'd love to hear about it. Rather than wait, I wrote up a small extension of @WebMvcTest to default these values.

First, an annotation for Spring to use in setting up a MockMvc (javadoc elided):

@Documented
@Import(JsonMockMvcConfiguration.class)
@Retention(RUNTIME)
@Target(TYPE)
@WebMvcTest
public @interface JsonWebMvcTest {
    @AliasFor(annotation = WebMvcTest.class)
    String[] properties() default {};

    @AliasFor(annotation = WebMvcTest.class)
    Class[] value() default {};

    @AliasFor(annotation = WebMvcTest.class)
    Class[] controllers() default {};

    @AliasFor(annotation = WebMvcTest.class)
    boolean useDefaultFilters() default true;

    @AliasFor(annotation = WebMvcTest.class)
    ComponentScan.Filter[] includeFilters() default {};

    @AliasFor(annotation = WebMvcTest.class)
    ComponentScan.Filter[] excludeFilters() default {};

    @AliasFor(annotation = WebMvcTest.class)
    Class[] excludeAutoConfiguration() default {};
}

Note it is a near exact lookalike of @WebMvcTest (minus the deprecated parameter). The important bits are:

  1. Marking this annotation with @WebMvcTest, a kind of extension through composition.
  2. Adding @Import to bind custom configuration to this annotation.
  3. Tying the same-named annotation parameters to @WebMvcTest, so this annotation is a drop-in replacement of that one.

Next a configuration class, imported by the annotation, to customize MockMvc:

@Configuration
public class JsonMockMvcConfiguration {
    @Bean
    @Primary
    public MockMvc jsonMockMvc(final WebApplicationContext ctx) {
        return webAppContextSetup(ctx)
                .defaultRequest(post("/")
                        .contentType(APPLICATION_JSON_UTF8)
                        .accept(APPLICATION_JSON_UTF8_VALUE))
                .alwaysExpect(header().string(
                        CONTENT_TYPE, APPLICATION_JSON_UTF8_VALUE))
                .build();
    }
}

Some points about this class:

  • @Primary is not necessary for Spring, but helped IntelliJ — perhaps I got lucky with Spring without @Primary, and IntelliJ highlighted a real problem.
  • It took quite a while to get defaultRequest(...) working. I was unable to (re)implement the relevant interfaces, and eventually found that passing any MockHttpServletRequestBuilder sufficed. Spring "merges" (overlays) the actual request builder from the test over this default, replacing POST and "/" with whichever HTTP method and path the test uses (eg, GET "/bob"). Only the header customization is used.

Example usage:

@JsonWebMvcTest(SomeController.class)
class SomeControllerTest {
    @Autowired
    private MockMvc jsonMockMvc;

    @Test
    void shouldCheckSomething()
            throws Exception {
        jsonMockMvc.perform(post("/some/endpoint")
                .content(someRequestJson))
                .andExpect(status().isCreated())
                .andExpect(header()
                        .string(LOCATION, "/some/endpoint/new-name"))
                .andExpect(content().json(someResponseJson));
    }
}

See the Basilisk project for source code and sample usage. (Basilisk is a demonstration project for my team illustrating Spring usage and conventions.)

Monday, December 21, 2015

RESTful helper script

I find this script useful working on modern RESTful services. It shows both the headers and the formatted JSON response body. The idea is that most times you provide a URL and want to see the full response. If you need extra flags for curl just add them (e.g., user/password). If you want to customize jq—say, filter for just a particular piece of the response—use a double-dash ("--") to separate curl and jq arguments:

An example with Spring Boot (plus some custom actuator endpoints). Note "jq" colorizes the output on the command line (below is plain text):

$ ~/bin/jurlq http://localhost:8081/remote-hello/health
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Application-Context: remote-hello:8081
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Mon, 21 Dec 2015 13:28:07 GMT

{
  "status": "UP",
  "cpu": {
    "status": "UP",
    "processors": 8,
    "system-loadavg": -1,
    "process-cpu-load": 0.22963892677420872,
    "process-cpu-time": "PT42.71875S",
    "system-cpu-load": -1
  },
  "file": {
    "status": "UP",
    "usable-disk": 55486464000,
    "total-disk": 299694551040
  },
  "java": {
    "status": "UP",
    "start-time": "2015-12-21T07:27:06.970-06:00",
    "uptime-beats": 0,
    "vm-name": "Java HotSpot(TM) 64-Bit Server VM",
    "vm-vendor": "Oracle Corporation",
    "vm-version": "25.66-b17"
  },
  "memory": {
    "status": "UP",
    "committed-virtual-memory": 991350784,
    "free-physical-memory": 1952854016,
    "free-swap-space": 1203003392,
    "total-physical-memory": 8540618752,
    "total-swap-space": 10354229248
  },
  "os": {
    "status": "UP",
    "arch": "amd64",
    "name": "Windows 10",
    "version": "10.0"
  },
  "threads": {
    "status": "UP",
    "count": 22,
    "daemon-count": 20,
    "peak-count": 22,
    "started-count": 26
  },
  "diskSpace": {
    "status": "UP",
    "free": 55486464000,
    "threshold": 10485760
  },
  "configServer": {
    "status": "UNKNOWN",
    "error": "no property sources located"
  },
  "hystrix": {
    "status": "UP"
  }
}

UPDATE: Tried Github Gist for the source, but it does not show in my blog feed reader. Here's the script:

#!/bin/bash

curl_args=()
for arg
do
    case "$arg" in
    -- ) shift ; break ;;
    * ) curl_args=("${curl_args[@]}" "$arg") ; shift ;;
    esac
done

jq_args=("${@-.}")

curl -s -D - "${curl_args[@]}" | tr -d '\r' | {
    while read line
    do
        case "$line" in
        '' ) echo ; break ;;
        * ) echo "$line" ;;
        esac
    done

    exec jq "${jq_args[@]}"
}

Saturday, December 19, 2015

Spring REST showcase

I've noodled for some time now at mini-projects showcasing Spring REST with as many bells and whistles as I could pack in before it broke. I've never reached a satisfactory conclusion, which is more a testament to my mercurial temperament than to Spring. My ideal project would include:

  • No XML, if avoidable (Lombok, I'm looking at you)
  • No properties files (thank you, Spring YAML support)
  • Annotations and code generation (thank you, Spring Boot and Lombok)
  • Good documentation (Swagger and RAML, why do you need to be so tricky?)
  • Complete REST adherence (HATEOAS, you are still an ugly child, sorry to say that)
  • Modern Java
  • No external container (Spring Boot to the rescue!)
  • Many example integration points (and this is why I stay with you Spring)
  • Maven build (sorry Gradle, I gave up makefiles so I would never again debug my build)
  • Production support (Spring actuator: genius; see OSI)
  • Cloud support
  • Full CD pipeline (Boxfuse, you may save me yet; Github and Travis, you're still the best)

Essentially I want to implement a showcase REST microservice adhering to Larry Wall's three great virtues of a programmer: Laziness, Impatience and Hubris.

Well, there's always yet another Github repo.

Updates I'll keep updating this post as I find new things to desire in a showcase project.

Saturday, June 27, 2015

REST API best practices

Vinay Sahni of Enchant writes Best Practices for Designing a Pragmatic RESTful API, a complete, current best practices article on REST APIs, chock full of explanation, examples and real APIs from top web sites. Enchant itself is a good example how to document a REST API. Vinay has been writing a long series of articles on REST APIs, plenty of chewy links for further reading. Main points:

  • Key requirements for the API
  • Use RESTful URLs and actions
  • SSL everywhere - all the time
  • Documentation
  • Versioning
  • Result filtering, sorting & searching
  • Limiting which fields are returned by the API
  • Updates & creation should return a resource representation
  • Should you HATEOAS?
  • JSON only responses
  • snake_case vs camelCase for field names
  • Pretty print by default & ensure gzip is supported
  • Don't use an envelope by default, but make it possible when needed
  • JSON encoded POST, PUT & PATCH bodies
  • Pagination
  • Auto loading related resource representations
  • Overriding the HTTP method
  • Rate limiting
  • Authentication
  • Caching
  • Errors
  • HTTP status codes
  • In Summary

Thursday, May 28, 2015

RESTful errors, Simple Boot

Handling Errors

Reading Travis McChesney's RESTful API Design Part III: Error Handling, there are more ways to represent errors in REST APIs than the returned body. A toy API I am writing as a playground for these idea is named Simple Boot, a jest on "Spring Boot". There I have an "X-Correlation-ID" header for tracking calls through services.

  • Should "X-Correlation-ID" be required?
  • Should it be supplied automatically?
  • When required and missing, what should the caller receive?

Answering the third the caller sees:

HTTP/1.1 400 Bad Request
Server: Apache-Coyote/1.1
Warning: 250 localhost:8080 "Missing X-Correlation-ID header"
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Thu, 28 May 2015 12:46:40 GMT
Connection: close

{
  "timestamp": 1432817125588,
  "status": 400,
  "error": "Bad Request",
  "message": "Missing X-Correlation-ID header",
  "path": "/hello/Bob"
}

Here rather than choosing among options, I take them all:

  1. Return a 400 for a bad request
  2. Describe the error in the response body
  3. Add a header describing the error

As McChesney points out, there is not general agreement on reporting errors. For example, Facebook returns 200 for errors, requiring parsing of the response body.

Using the "Warning" header for this purpose is uncommon but supported in the specification albeit obliquely. However not everyone agrees. I use warning code 250, one I made up. The 1xx codes are transient errors; the 2xx codes permanent. In another example I use a 1xx code for an upstream service currently unavailable, and return cached data, which is closer to the described uses of "Warning".

Other header options:

  • Using a custom code outside 1xx or 2xx with "Warning". This makes sense for in-house services, may cause issues with caching devices but unclear
  • Use a custom HTTP header, a common solution, again may have issues with intermediate devices

Some Spring Bonus

Simple Boot has been fun and instructive. One Spring Boot feature I stumbled on is automated binding of configuration properties. For services requiring "X-Correlation-ID" I configure with @ConfigurationProperties(prefix = "headers.correlation-id.server"). To automate clients I use @ConfigurationProperties(prefix = "headers.correlation-id.client").

As an example of solving one issue with headers, I write Feign pass-through support.

Of course there are tests.