Simple API example for demux application

in eosio •  5 years ago 

1_uHzooF1EtgcKn9_XiSST4w.png
Img source

In my previous post, I talked about the demux example application.

API

This simple example allows you to get a list of transactions captured and stored in the database, using the demux application described in the previous article, using the REST API.

Unfortunately, due to the fact that the example is simplified for security reasons for sensitive information, there are 2 identical routes. Although it may be better to understand how to modify an example to fit your needs simply by changing one of the routes, models and so on.

Example

The code can be found here: https://github.com/4ban/EOS-demux-js-api-example

Details

For simplicity, but at the same time so that the example can be easily modified, for example, for different document models in the database, I left 2 identical routes.

app.use('/col1', collection1Router);
app.use('/col2', collection2Router);

The Mongoose (MongoDB) schema should be the same as that used in demux service. There is nothing new.

const mdl1Schema = new Schema({
    from: String,
    to: String,
    quantity: String,
    currency: String,
    memo: String,
    trx_id: { type: String, unique: true },
    blockNumber: Number,
    blockHash: String,
    handlerVersionName: String
})
mdl1Schema.index({ from:1, blockNumber: -1 })
mdl1Schema.index({ to: 1, blockNumber: -1 })

mdl1 = mongoose.model('mdl1collection', mdl1Schema)

The route is pretty simple too. The code is quite standard, implements the simplest pagination. There are also checks to prevent negative numbers from being used.

Pagination (i.e., use the skip and limit parameters during the database requests) is needed so that the query time does not exceed 100ms as well as for ease of use the API in applications.

Next comes the function that will query the database and return the data for display. In this case, the function getEOSHistoryAccount1 is called (i.e. function type 1).

If you need to use another collection, other fields when searching and generally change the data retrieval function, this is easy to do by analogy with route col2 and the function getEOSHistoryAccount2.

router.get('/:accountName', (req, res) => {
  var skip = parseInt(req.query.skip) || parseInt(process.env.SKIP)
  var limit = parseInt(req.query.limit) || parseInt(process.env.LIMIT)
  if (skip < 0 || limit < 0) {
    var response = {
      "error": true,
      "message": "Invalid skip or limit number, should start with 0 or 1 respectively"
    };
    return res.send(response)
  }
  services.History.getEOSHistoryAccount1(req.params.accountName, skip, limit).then(function (result) {
    return res.send(result)
  }).catch(function (err) {
    var error = {
      "error": true,
      "message": err
    }
    res.send(error)
  })
})

The function makes a request with the help of Mongoose, making a selection by fields from and to in deccending order.

Compound indices are needed to make this possible at an acceptable speed.

Screen Shot 20190516 at 8.44.03 PM.png

function getEOSHistoryAccount1(accountName, skip, limit) {
  return new Promise((resolve, reject) => {
    var response = {}
    models.mdl1.find({
      $or: [
        { 'from': accountName },
        { 'to': accountName }
      ]},
      null,
      {
        skip: skip,
        limit: limit,
        sort: { blockNumber: -1 }
      }, function (err, transactions) {
      if (err) {
        var error = {
          "error": true,
          "message": err
        }
        reject(error)
      }
      response = {
        "error" : false,
        "data" : transactions
      };
      resolve(response)
    })
  })
}

API examples:

MethodURLDescription
GET/List of available API
GET/col1List of available APIs for account 1
GET/col1/{accountName}History for account 1. First 10 transactions
GET/col1/{accountName}?skip=10History for account 1. Second page (i.e next 10 transactions)
GET/col1/{accountName}?limit=100History for account 1. First 100 transactions
GET/col1/{accountName}?skip=100&limit=100History for account 1. Second page (i.e. next 100 transactions)
GET/col1/stats/{accountName}Stats for account 1
GET/col2List of available APIs for account 2
GET/col2/{accountName}History for account 2. First 10 transactions
GET/col2/{accountName}?skip=10History for account 2. Second page (i.e next 10 transactions)
GET/col2/{accountName}?limit=100History for account 2. First 100 transactions
GET/col2/{accountName}?skip=100&limit=100History for account 2. Second page (i.e. next 100 transactions)
GET/col2/stats/{accountName}Stats for account 2

Summary

I hope I managed to get the message across and make the right example.

I do not have much experience writing such things. But I tried my best to test the performance with the help of JMeter tool.

Results

There were not enough resources on my machine to imitate more than 4000 people.
On the server, 2 different versions of API were launched, as well as a MongoDB server which was constantly being accessed with a demux service from another server. So these tests are hardly accurate. The server is Amazon AWS instance t2.xlarge.
Screen Shot 20190518 at 9.25.55 PM.png

Simulated usersSamplesAPDEX IndexAPDEX statusCPU loadMEM loadAVG response time
100030000.999Excellent2%0.6%35.79ms
4000120000.999Excellent4%0.6%34.99ms

56982160a45278006b4e11e989c9ac80fe32d0cb.png

More details from JMeter:

56983575f47f09806b5111e9908b765850b5f256.png

56983576f47f09806b5111e99b43e13813dc4177.png

56983577f47f09806b5111e999589d3d98daa7cb.png

Installation

MONGODB_URL=mongodb://{server_address}:27017/DBname
PORT=3000

Default pagination parameters

SKIP=0
LIMIT=10

Credentials

MONGODB_USER={username}
MONGODB_PASS={password}

Usage

To run the server: npm start

Links

Demux

EOF

Enjoy. Git cool!

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Congratulations @ban2ban! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

You made more than 10 comments. Your next target is to reach 50 comments.

You can view your badges on your Steem Board and compare to others on the Steem Ranking
If you no longer want to receive notifications, reply to this comment with the word STOP

To support your work, I also upvoted your post!

Vote for @Steemitboard as a witness to get one more award and increased upvotes!