Building a MicroService APIs using the GateWay Pattern

Intro.

I will assume that you know the theory of MicroServices, and i will assume that you need a hands on reference aid on how to build a Complete MicroService Based REST APIs utilizing the benefits of the concept and also simple enough to build over.

Values we seek from MicroService 

  1. Build as much independent raw functioning services fulfilling the whole domain functionality.

  2. Making sure that every Service is running and maintained separately without any crack due to another service defect.

  3. Providing a separate deployment ready services, the thing that will aid on managing the load upon certain service separately with MAX. cost control.

  4. Supporting any combination of workflows using the MicroServices without the need to change the Services implementations.

I use a GateWay for MicroServices communication

The alternative is “Point to Point” communication, where the services can communicate with each other for sake of certain workflow or dependencies, but the issue about this technique is that we are no fully satisfying the Services dependencies and highly coupling the services stability on each others, and further more the Point to Point configuration is a nightmare specially when we have many domains.

The Design Essentials

  1. Every MicroService represents a different domain with standalone and self concerned business logic.

  2. Every MicroService doesn’t has aware enough with the other MicroServices end points of even functionality, so MicroServices direct communication is totally prohibited.

  3. Every MicroService is a REST API with unique EndPoint and could be hosted separately with unique security configurations.

  4. GateWay is a REST API with unique EndPoint.

  5. GateWay is the only EndPoint exposed to the FrontEnd Applications, so Microservices are only accessible by the GateWay.

  6. GateWay is the the place to build specific business workflow using the different MicroServices.

  7. Finally, and i haven’t seen any one apply this role 100%, is to make sure that every MicroService using its own DataBase and DataModels with any need to data from other Services DataBases, this is perfect to keep improving you Design to fulfill this concept and to save the separation down to all layers.

Sample Generic Business Practice

We need to build a Microservices to handle a TimeSheet System functionality.

we will need the following MicroService:

  1. UserAPI (Host In http://ip:port/api)
    1. PostNewUser        http://ip:port/api/post
    2. PutExistUser         http://ip:port/api/put/id
    3. DeleteExistUser   http://ip:port/api/delete/id
    4. GetAllUsers           http://ip:port/api/get
    5. GetUserById          http://ip:port/api/get/id
  2. MessagingAPI (Host In http://ip:port/api)
    1. GetAllMessages
    2. PostMessage
    3. DeleteMessage
    4. PostReply
  3. VacationAPI
    1. PostNewVacation
    2. DeleteNewVacation
    3. GetAllVacations
    4. PutExistVacation
  4. TimesheetAPI
    1. PostNewTimeSheet
    2. PutExistTimeSheet
    3. GetAllTimeSheets
  5. ReportingAPI
    1. GetReportDataById
  6. ProjectAPI
    1. PostNewProject
    2. PutExistProject
    3. DeleteProject
    4. GetAllProjects
    5. GetProjectById
    6. AddUserToProject

Now, we have 6 Separate and independent MicroServices hosted separately in different endpoints (ip/port).

Now we need to build another REST API represents the GateWay handelling the business Workflows to call from the FrontEnd Application (DeskTop APP., Web. APP. , Mobile, another Service , …etc).

 

e.g. Build GateWay REST function to Create Project and Add all users to it.

Steps In GateWay :

  1. Create Project by calling the ProjectAPI function PostNewProject

  2. Get All User by calling the  UserAPI function GetAllUsers

  3. Enumerate on the Users and Call The ProjectAPI function AddUserToProject.

  4. Return Operation Status

This Function should be a Post function expects [From Request Body] an object of Project, and the GateWay function should respond with Status (200, 404, 400, 500, …etc) according to how you handle the API errors.

so at the end , the FrontEnd application communicates with an API like http://gatewayip:port/api/postprojecttoeveryone

Important Notes

  1. The Microservices changes cadence should be minimal as it provides a row fundamental CRUD operations over specifuc business model.

  2. Every business Application with different requirements require New GateWay and not new MicroServices.

  3. Exception Handling and Error Pulling should be in a way to stop the technical inner exception on the layer of GateWay and Log It, and send Friendly helpful messages with Error Code as a GateWay responce.

Advertisements

Alternatives to build todays Web Applications and different hosting choices

Dears,
While we are so confused with What to follow for the optimal Architecture and what tool to use and what is the best integration to build a stable and responsive WEB application. and also where to host for stable accessibility. i have defined three alternatives for building a Web APP. with optimal hosting for each.

Note that you can comment with more alternatives and i promise i will update the post directly.

Alternative 1: For Mission Critical Apps

DataBase : SQL Server or ORACLE

Server Side APIs :

  1. Entity Framework  : For Database object representation and Database Operation management
  2. ASP.net WEB APIs : For Fast and Simple REST APIs.
  3. WCF Services : for Dual Binding and Custom Security.

Data Binding and Manipulations :

  1. AngularJS 1.x : If your UI is Template Oriented and no UI Dynamic creation expected.
  2. ReactJS : If you you seek a Script First Data Driven APP, and if your UI is highly Dynamic upon the Data with different Conditions.
  3. ReactJS : If you want to create a WorkFlow.
  4. D3JS : For Fancy SVG Charts, Comparisons, Interactive and Organized and Dynamic Data Representations.
  5. ThreeJS : If you want to add 3D or Animation.

Front End Business logic :

  1. JavaScript : If you and your team have no problems with Heavy Scripting and not fully customized Object Oriented code.
  2. ECMAScript : If you are like the above but need to write less code and more structured and readable, and want to keep the Un-Typed Scripting features (variables are not typed, all vars).
  3. Type Script : for sake of minimal code, as much like C# and Java, Fully Object Oriented with interfaces, for building modules with access modifiers (public, private , ..etc), and for Type and Build  Error Check in Compile Time.
  4. CoffeScript : Just like the ECMA but wants to witie RUBY or PYTHON like Code with space separation for code block.

Host : Windows Server

Alternative 2: For Non-Mission Critical and Huge Data Manipulation

DataBase : MongoDB, Cassandra (I recommend Mongo)

Server Side APIs :

  1. NodeJS : For Database object representation and Database Operation management.
  2. ASP.net WEB APIs : REST APIs
  3. WCF  Services : for Dual Binding and Custom Security.

Data Binding and Manipulations :

  1. AngularJS 1.x : If your UI is Template Oriented and no UI Dynamic creation expected.
  2. ReactJS : If you you seek a Script First Data Driven APP, and if your UI is highly Dynamic upon the Data with different Conditions.
  3. ReactJS : If you want to create a WorkFlow.
  4. D3JS : For Fancy SVG Charts, Comparisons, Interactive and Organized and Dynamic Data Representations.
  5. ThreeJS : If you want to add 3D or Animation.

Front End business logic :

  1. JavaScript : If you and your team have no problems with Heavy Scripting and not fully customized Object Oriented code.
  2. ECMAScript : If you are like the above but need to write less code and more structured and readable, and want to keep the Un-Typed Scripting features (variables are not typed, all vars).
  3. Type Script : for sake of minimal code, as much like C# and Java, Fully Object Oriented with interfaces, for building modules with access modifiers (public, private , ..etc), and for Type and Build  Error Check in Compile Time.
  4. CoffeScript : Just like the ECMA but wants to witie RUBY or PYTHON like Code with space separation for code block.

Host :

  1. DataBase Server : Windows Server or Linux (I recommend)
  2. Application Server : IIS over Windows in case of WCF or WEB APIs , Linux for the NodeJS.

Alternative 3: For Cloud Ready Application

DataBase :

  1. Create a Cloud VM [Windows or Linux – Linux is better for low prices](From AZURE or Google Compute pr AMAZON) and install mongoDB.  (NB. MongoDB required High Memory Machine so focus on the RAM).
  2. Use AZURe mongoDB service or Google Mongo Service, that is more scaleable and will take you out of the headache of Database Server management and Configuration, but much more costly compared with the last option.
  3. Store Your Data Over Cloud Provider Storage Service (I recommend AZURE Table Storage for Tabular Data and AZURE Blob Storage for File Storage) NB. Revise this post before using The Table Storage.

Server Side APIs :

  1. NodeJS : For Database object representation and Database Operation management.
  2. ASP.net WEB APIs : REST APIs
  3. WCF  Services : for Dual Binding and Custom Security.

Data Binding and Manipulations :

  1. AngularJS 1.x : If your UI is Template Oriented and no UI Dynamic creation expected.
  2. ReactJS : If you you seek a Script First Data Driven APP, and if your UI is highly Dynamic upon the Data with different Conditions.
  3. ReactJS : If you want to create a WorkFlow.
  4. D3JS : For Fancy SVG Charts, Comparisons, Interactive and Organized and Dynamic Data Representations.
  5. ThreeJS : If you want to add 3D or Animation.

Front End business logic :

  1. JavaScript : If you and your team have no problems with Heavy Scripting and not fully customized Object Oriented code.
  2. ECMAScript : If you are like the above but need to write less code and more structured and readable, and want to keep the Un-Typed Scripting features (variables are not typed, all vars).
  3. Type Script : for sake of minimal code, as much like C# and Java, Fully Object Oriented with interfaces, for building modules with access modifiers (public, private , ..etc), and for Type and Build  Error Check in Compile Time.
  4. CoffeScript : Just like the ECMA but wants to witie RUBY or PYTHON like Code with space separation for code block.

Host :

  1. DataBase Server : as illustrated in the DataBase part
  2. Application Server :

If you use WEB APIs or WCF : Use AZURE APP Service for easier and faster updates or indeed Windows VM.

If you use NodeJS you can host in Linux VM or Windows VM.

General Notes From My Experience:

  1. NodeJS is implemented over Chrome V8 Engine (compiler), so it supposed to be faster that the WEB APIs and indeed the WCF.
  2. WCF Service is best choice when it comes to Dual HTTP Binding with Custom Security Configurations.
  3. AZURE Cloud is easier than Google compute when it comes to Scaling Up/Down.
  4. Google Compute Engine is much more Lower in price than AZURE even for Windows Machines.
  5. Google Compute Engine team might suspend your VM for days if they felt that you are misaligned with the Terms Of Use , and they might re open when they discover that they were wrong !!
  6.  Angular2 will introduce the Dynamic Component like the ReactJS with much more better performance than Angular 1.x.

Useful Links Regarding the Above:

Click For more Info about how to install and Configure MongoDB in Windows.

Click For more Info about how to install and Configure MongoDB in Linux.

Click For more Info about how to create a REST API using node JS over MongoDB.

Click For Getting Started with ASP.net WEB APIs

Click For Angular Official website

Click for ReactJS Official website

Click for D3Js Official website

Click for ECMAScript Official website

Click for TypeScript Official website

Click for ThreeJS Official website

AZURE limited 64KB table storage entity property ( Solutions )

Hi,
If you are new in windows azure and started to use AZURE Table Storage and inserting entities to specific table partition ..

you might get panicked if you tried to insert big data string and you got “BAD REQUEST” or error in inserting the Entity successfully.

its okay after all, but there are some roles while storing data to AZURE table storage like the RowKey formatting where you shouldn’t add ‘#’ to the RowKey and it should be Unique .. etc. but its not our concern now, where we focusing on the Single Entity Property Max size is 64KB.

Lets make a simple practice to understand where the EPIC is.

Copy the String you are trying to insert as a property value and save it in a text file and check the .txt file size ! .. the result is that the size is bellow 64KB,but you will continue receiving the same error.

The Dark fact here is that ;

The Table Storage Property Size is measured as :

Prop. Size = 4 bytes + Len (Partition-Key + Row-Key) * 2 bytes + For-Each Property(8 bytes + Len(Property Name) * 2 bytes + Sizeof(.Net Property Type))

Source : here

this means the Prop. size almost duplicated before saving, and this is a nightmare indeed if you are saving a URIs or big text. whatever the value of saving such big data to Table Storage, but its okay no one should attack you for this. any ways .. here is my solutions:

Solution 1: Divide

1. Divide the Big Prop. Data to smaller parts according to the Average size or you might make it dynamic to detect the Total Size and Assume the accepted size and Divide Total / Assumption = [the ‘N’ you should divide to].

2. Add a PartNo Prop. to use farther when you retrieve the data to merge the parts together in one value.

Solution 2: Use Blob Storage

1. Blob storage is here for saving files, and for your case you will use Text File to save your big Prop. Data that exceeds the 64KB to the file.

2. save the file to the Blob Storage and Keep the URL of the file to set as the Original Table Storage Prop. value.

3. While you are Retrieving the Data, you will do kind of mapping to Get the Entity from the Table Storage, then map. Get the File Data based upon the URL saved in the Table Entity.

I believe in Solution 2 rather than Solution 1 as it uses the power of AZURE Blob Storage in Sync. with Table Storage. Enjoy.

Solution 3: Use MongoDB service in AZURE

This is indeed more expensive but it will get you out of the Limited Prop. size.

Solution 4: Buy a VM and install your Own MongoDB

This is much less expensive than the last solution but the point here that you can always decrease the Cost of your VM by changing the operating System from Windows to Linux.

To study more about installing MongoDB in Windows of Linux , check my two posts (For Linux, For Windows).

Note that Mongo DB is a FREE NO-SQL Database Engine and what ever you programming technology you will find a driver for.