Monday, December 28, 2015

Send messages using AMQP in Java in Azure Event Hub and use Azure Stream Analytics to read and save in Azure Storage


Steps:

1. Create, and configure, and write code to send messages using Azure Event Hub:


Sample json message:

{
   "time":"12-27-2015:23:53:372 -0800",
   "code":"1984567839209"
}

2. Create Azure Storage and container

Create SA in classic aka v1 portal. For easy on-screen configuration. You could also try with v2. Either way it works great.

3. Create Azure Stream Analytics job, use the above SA created above.

Follow the steps mentioned in the href in the Reference section.

4. Configure Azure Stream Analytics with input as Event Hub and Azure Storage as output. Configure a query.

Follow the steps mentioned in the href in the Reference section.

Start with a sample query:
SELECT
*
INTO
[<sa account/output name>]
FROM
[<event hub>]

Please note that the square brackets remain. Replace all of the angle bracket along with your configuration.

5. Start the job

Follow the steps mentioned in the href in the Reference section.

6. Send messages (json with UTF-8)

Follow the href provided above in step 1.

7. Verify that the message has landed in storage blob container

Whichever tool you have, validate that the content has been pushed by ASA to SA blob container.


8. Check stats of Azure Stream Analytics job – storage account


9. Check stats of Azure Stream Analytics job - metrics

Check it under Monitor section.

10. Fix errors

e.g. if the json format is bad, etc. the message should be ample clear J

Act accordingly. E.g. test out the input using ASA test tool.

Stop the job. Fix the error. Start the job.



No comments:

Post a Comment