Anuvadak (अनुवादक) - New features

Repository

https://github.com/node-muneem/anuvadak

I've added some new features to Anuvadak.

Project summary

Anuvadak is an opensource project which adds serialization, and body parsing methods to muneem web framework.

Change summary

In the current change, I've added/updated the methods to parse request stream to read XML, multipart forms, URL encoded forms and files. I've also added/updated the methods to write text, js objects as XML. All the methods can be used in the following way;

const muneem = require('muneem')();
const anuvadak = require('anuvadak');
muneem.use(anuvadak.XXX, globalOptions);

muneem.add('handler', function(asked, answer){
        await asked.readXXX();
        //..
    answer.writeXXX( data );
})

Commits

• https://github.com/node-muneem/anuvadak/commit/4751342b9b574d37fcd3c837ff8c9c5ce9740218

Change detail

Muneem web framework supports core functionality to read text information from the request stream and write text data to respond. All the methods added in this library are to hide details from the user and provide simple to use methods.

In the previous release, Anuvadak was adding all the methods internally, where the user was losing the options to skip particular type of serializer or body parser.

In previous release

const muneem = require('muneem')();
const anuvadak = require('anuvadak');
muneem.use(anuvadak, combinedGlobalOptions);

muneem.add('handler', function(asked, answer){
        await asked.readXXX();
        //..
    answer.writeXXX( data );
})

After current release

const muneem = require('muneem')();
const anuvadak = require('anuvadak');
muneem.use(anuvadak.XXX, globalOptionsForX);
muneem.use(anuvadak.YYY, globalOptionsForY);

muneem.add('handler', function(asked, answer){
        await asked.readXXX();
        //..
    answer.writeXXX( data );
})

It has also made the library simple and clean: https://github.com/node-muneem/anuvadak/blob/master/src/anuvadak.js

XML

XML read and write methods are changed to use new changes in the library. This change is more about using XML specific options instead of extracting relevant options from combined global options. I've used Fast XML Parser so the parsing can be customized as per the user's choice.

URL encoded form

This is the new feature added to read the URL encoded forms. I used qs npm package to parse the query string and extract form detail.

function UrlFormReader(globalOptions){
    if(!globalOptions) globalOptions = {};

    this.readForm =  function(options){
        options = Object.assign({}, globalOptions, options);

        if( !this.queryStr || (options.confirmHeaderParam  && this.headers["content-type"] !== "application/x-www-form-urlencoded" ) ){
            return;
        }
    
        this.body = qs.parse( this.queryStr, options ); //this.body will be object now
        return this.body;
    }
}

Checking header parameters only on demand speed up the processing. Here is the code: https://github.com/node-muneem/anuvadak/blob/master/src/urlForm.js

Form Handler

Form handler is a wrapper around formidable npm package. A user can use this to avoid any complex configuration. This function uses metered request stream set by the Muneem framework. Hence safe to use. Here is the code summary. Some code is replaced with comments for explanation purpose;

const globalOptions = buildOptions(defaultOptions, globalOptionsInput);

//build directory path

this.getInstance = function(optionsInput){
        const options = buildOptions(globalOptions, optionsInput);

        //set upload path

        const form = new formidable.IncomingForm();
        //set form options
        
        form.read = () => {
                form.parse(this.stream);
        }

        return form;
}

This is how it can be used in any request handler;

const muneem = require('muneem')();
const anuvadak = require('anuvadak');
muneem.use(anuvadak.form, globalOptionsForForm);

muneem.add('handler', function(asked, answer){
        var formHandler = asked.getFormHandle(options);

    formHandler.on("field", function(name, value) {
        //..
    });

    formHandler.on("file", function(name, file) {
        //..
    });

    formHandler.read();

        //..
})

Here is the code: https://github.com/node-muneem/anuvadak/blob/master/src/formHandler.js

Form

This feature allows the user to read the form data from the request stream without handling any event of formHandler. A user can configure if the files need to move to a particular location, what if the same file is uploaded multiple times, or if it is larger than expectation etc. Here is the code for the same. I've replaced some part with comments for explanation purpose.

The full code can be found here: https://github.com/node-muneem/anuvadak/blob/master/src/form.js

const form = formHandler.getInstance.bind(this)(options);
//set form options

this.body = {};
form.on('field', (fieldName, value) => {
        if(!options.field.on.receive){
                this.body[fieldName] = value;
        }else{//user want to handle it
                options.field.on.receive(fieldName, value);
        }
});
form.on('file', (fieldName, file) => {
        if(options.file.on.receive){//if user want to handle it
                options.file.on.receive(fieldName, file);//let him handle
        }else{//otherwise move to user defined location
                this.body[fieldName] = file;
                let saveTo = options.file.saveTo ; //if present then move and rename file to given location
                if(!globalOptions.file.saveTo && !saveTo){
                        //do nothing
                }else{
                        if( !isDirectoryExist(saveTo) ){
                                saveTo = path.join(globalOptions.file.saveTo, saveTo);
                                if( isDirectoryExist(saveTo) ){
                                        saveTo = "" ;
                                }
                        }
                        if(saveTo){
                                file.newPath = path.join(saveTo, file.name);
                                if( isExist(file.newPath) ){
                                        options.file.on.duplicate(fieldName, file, saveTo );
                                }else{
                                        fs.rename(file.path
                                                , path.join(uploadTo, options.file.on.name(file.name) )
                                                , options.file.on.move );
                                }
                        }else{//if user has defined the wrong location
                                options.on.error(new Error(`Invalid location to save file: ${saveTo}`));
                        }
                }
        }

});

Now I've used Promise so that the data can be used by the response handler.

        await new Promise( (resolve, reject) => {
            form.parse(this.stream);
            form.on('error', (err) => {
                reject(err);
                options.on.error(err);
            });
            form.on('aborted', () => {
                reject();
                options.on.aborted();
            });
            form.on('end', () => {
                resolve(this.body);
                options.on.end();
            });
        });

Files

Reading files from the request stream is as similar to reading forms in the above approach. However, reading only files with the small change in the code can improve the performance. Hence I've created a separated method for that. It seems more meaning full.

Here is the full code: https://github.com/node-muneem/anuvadak/blob/master/src/files.js

if(options.sync){
        await new Promise( (resolve, reject) => {
                form.parse(this.stream);
                form.on('error', (err) => {
                        reject(err);
                        options.on.error(err);
                });
                form.on('aborted', () => {
                        reject();
                        options.on.aborted();
                });
                form.on('end', () => {
                        resolve(this.body);
                        options.on.end();
                });
        });
}else{
        form.on('error', options.on.error);
        form.on('aborted', options.on.aborted);
        form.on('end', options.on.end);
        form.parse(this.stream);
}

In addition to that relevant unit tests were added.

Contribution

I'm planning to add serializers and body parsers for protobuf, nimn etc in future. If you're interested please support, fork, and suggest improvements.

GitHub Account

https://github.com/amitguptagwl