Data Service Feature: Using the Incoming Request Capture Wizard
This article examines how to use the Incoming Request Capture Wizard to rapidly create new Services and Rules to easily integrate Maque into existing projects.
For more detailed information about Data Services, Methods, Rules and Arguments please refer to these other articles:
The Incoming Request Capture Wizard
Integrating Maque into existing projects can sometimes be a challenging and time consuming process. Often your project already leverages existing services, which drive the application logic. To integrate Maque into an existing project you will need to recreate all of the service API behaviors within Maque, before you can add Maque to your development and testing workflow. Prior to the Incoming Request Capture Wizard (IRCW), this process required you to create each Service, Rule and Argument one-by-one based on the existing API signature. Once the Rules are created, you will need to produce response data for the Rules to return. The process of creating each and every Rule via the UI can become tedious when faced with many different APIs and scenarios. With the IRCW, you can now enable your application and existing APIs to drive the creation of Rules quickly and automatically.
Using the Incoming Request Capture Wizard
The IRCW can generate Services for you in two different ways: as a responder or as a proxy. The first way is as a basic responder. You define a default HTTP response (such as 200 OK) and any incoming request is tracked by the IRCW and replied to using the default HTTP response you selected. Once all the requests you need Services for are captured, you have the IRCW convert them into Services, Methods, Standard Rules and Arguments. This allows you to use an existing application with data calls to create the Rules for you. This can be helpful if you prefer to write the service calls in your application first, before the real services exist.
The second way to use the IRCW is as a proxy that forwards all incoming requests to an existing service and then passes the services responses back to the requester. Every request and response is tracked by the IRCW. You can then select which request/response pairs you want to use to create Services, Methods, Proxy Rules and Arguments. Unlike the previous way of using the IRCW as a responder, the proxy methodology will create Proxy Rules and cache the responses that where returned by the service. This workflow is incredible powerful for quickly capturing the current application and services and allowing you to easily integrate Maque into your existing projects.
Starting the Incoming Request Capture Wizard
To use the IRCW you first need to have a project open. It can be a new project or an existing project. Once you have a project open, you can start the IRCW by using the menu Data Services > Start Incoming Request Capture Wizard... or CMD-I (mac) or CRTL-I (windows).
The IRCW dialog opens and will automatically start the HTTP server if it has not been already been launched. Once the IRCW is started all incoming requests will be handled by the IRCW and any existing Services and Rules will be ignored until the IRCW is closed. Once the IRCW is started you can configure it as either a responder or as a proxy. At this point, you can point your application service calls to Maque's HTTP Server URL (default: http://localhost:5656).
Using the IRCW as a responder
To use the IRCW as a responder, you select Default Response Type and choose the response you would like Maque to return. By default, the IRCW is configured to return 200 OK. Unlike a standard Rule, you can only set the response type, you can not define the returned data. While the IRCW is open any incoming request will be accepted, tracked by the IRCW and the response we be returned based on your response type selection.
Using the IRCW as a proxy
To use the IRCW as a proxy, you select Use Exiting Service and define the base URL for the the service API. Unlike a Proxy Rule, where you need to define the exact end-point for the URL, the IRCW URL should only be the protocol, host and port, if required (ex: http://www.myservice.com/ or http://www.myservice.com:8080/). When an incoming request is captured by the IRCW, the IRCW will extract the path and arguments and generate the outgoing request replacing the Maque hostname with the provided hostname. For example: http://localhost:5656/users?id=10 -> http://api.myservice.com/users?id=10
The IRCW will track both the request and the response for each call. This will allow the IRCW to generate a series of Proxy Rules using the response and URLs based on the request/response pairs.
Inspecting requests and responses
As incoming requests are tracked (either in responder or proxy mode) you can inspect each request's header, body, response header & body and the captured arguments. To view the content, select the request in the list and then you can choose the different tabs to view the details. This allows you to verify the request and response match the structure and content you expect.
Filtering requests and responses
The IRCW provides the ability to search and filter the requests and responses listed in the grid. This allows you to quickly find all the requests that use the a specific url or return type, such as 304 Not Modified.
Selecting requests to create Services, Rules and Arguments
Once you have captured all the requests that you want to convert into Services, Methods and Rule you can select all of the requests or cherry pick each request. Once you have selected one or more requests you can then select Save and Maque will generate the services for you.
Creating Services and preventing duplication
Once you have selected the request(s) to generate Services and Rules and selected Save, Maque will process the selected requests to determine what to create. The goal of the IRCW is to intelligently create Services and Rules so that no duplication occurs. The IRCW does this by creating new Services and Rules only if they are unique.
The first step of the generation is to determine if any of the selected requests are duplicated, such as making the same request twice from the application. Only unique requests will be used to generate Services and Rules. Once the request set has been processed for duplicates, the IRCW looks at the path for each request (ex: /users). The path is then compared to any existing Services in the project. If the IRCW finds a matching Service path, then it will use this Service as the parent of any matching request, otherwise a new Service will be created.
Once the Service is defined (either new or existing) the IRCW matches the requests method type (GET, POST, PUT, DELETE). The IRCW then looks at the request's argument type (form encoded, json, or xml) and determines if any existing Rules for the Method type match the incoming request's arguments. If none of the Rules match, a new Rule is created, otherwise this request is skipped because a Rule already exists. Please note that RegEx Rule Arguments will not be tested to see if they capture the incoming request, only the exact name and value are compared.
When a Rule is created, the Rule type is based on the mode the IRCW is in. If the IRCW is in responder mode (using default responses) then the generated Rules will be Standard Rules. If the IRCW is in proxy mode then the Rules will be Proxy Rules with the proper URL path and the response cached for future use. After the Rules are created, Maque will show a report dialog informing you how many new Rules where created, how many of the selected requests where duplicates and if any of the requests matched existing Rules.
Limitations
Cached response types
When caching requests, make sure that the server is returning the correct response. Sometimes, the server will return a 304 Not Modified response stating that the data is unchanged. If you capture this response the body will have no data, because the response is simply telling the browser to use its own cache because the content has not changed. Later, when the content in the browser's cache expires, this will cause an issue because the Maque server will not be sending back data. If this occurs, you will need to clear the Maque Proxy cache and let the server return the proper data.
Proxy and authentication
The IRCW has some limitations based around the kind of services that it can proxy to. Services that rely on OAuth or other custom hash based signatures of the request and timestamp may not work during the authentication process. These limitations usually occur with server to server requests. For general authentication, such as Rails origin security, cookie based sessions and other header driven authentication we recommend using the Project's Proxy Forward feature to have all requests routed through Maque and then forwarded to the endpoint. This prevents relying on Cross Origin (CORS) support and will properly attach sessions to the inbound and outbound requests.