Front-End (Web and Mobile) development workflow

Sharing here my time-tested agile front-end development methodology I have been using in all my greenfield development projects with success with my team comprises of:

  • UI/UX Engineers with business and creative acumen
  • HTML programmers
  • Front-End engineers experienced in building Angular and React.JS mobile and web applications of any size
  • REST API (Backend) developers
  • DevOps engineers.

I pictorially depicted the development process. It has 5 major steps

Web & Mobile App Workflow

  1. Entire application visuals, pages, forms etc. etc. created in Invision with exactly how we wanted the finished application would look like with the ability to navigate each screen by clicking on action buttons as-if we are using the real application. We can collect feedback on each screen within the invision with comments coach marks.
  2. Once the visuals are approved, the base application or skeleton application built in clean HTML and CSS by professional HTML programmers. You need highly skilled HTML programmers, not the ones who can create an APP using Twitter bootstrap or copycats.
  3. Then, build React.JS and REST API simultaneously
    1. Create the React.JS or Angular.JS application using the just created HTML and CSS templates and wire-up each screen with mock JSON Data
    2. Build the REST APIs using the programming language and framework of our choice. I use Node.JS (Sails.JS) or Java (Spring) or Python (Flask) as lambda functions and exposes it thru API gateway. Before the actual development, we finalize every endpoint mapped to the user interface with mock data leveraging integration response in AWS API Gateway. We also use Mockaroo or Facker.js to generate synthetic data to mimic production data.
  4. then, integrate the Front-end and Back-End and continue to deploy and test the application. 90% of the time, there will  (should)  not be any changes in the REST Endpoints between mock and actual REST API. It should be a black box for the front end developers.
  5. Go-live from the day we completed step # 2. Our DevOps automate the build and deployment process and release the packaged code as docker image as well as host it in AWS S3. We create 3 main environments with various configurations (we follow conventions over configuration approach)
    • Development – Every day or alternate days we release current dev. branch version with new web links without destroying previous dev. releases
    • Staging – Every week we release staging deployment for the client to test and collect feedback
    • Production – Case by case basis, we release production version when we do Staging release to make sure the code works in production as expected without any last minute hassles.

 

BUDDY-WORKS-logo-blue

We Use Buddy Works for Continuous Integration workflow from Github code pull to Build to package, version bump, deploy, back-up & roll-back, slack notification and sending emails the build URL to project team members and stakeholders.

 

createlyNote: The diagram I used in this post is created using creatly. You can get the original editable diagram from my GitHub as part of giving back to the opensource community.

 

 

 

 

 

Inbound Email processor using Node.JS Mail-Notifier

I have written a simple but full fledged inbound email processor application using mail-notifier npm which internally uses imap & mailparser. The complete source code is in my Git Repository for download or clone

image

[Note: Image created using Lucid chart online diagramming tool]

In this application, I have taken care of the following business functionalities

  1. Continuously look for all incoming email messages
  2. When email arrived, process it and form a modified email JSON object with the following properties
    • From
    • To
    • Subject
    • Plain Text Body
    • Message ID
    • Date of the email
    • Attachments
  3. Keep the configurations in a separate config file i.e. Email credentials, Mongo connection string etc.
  4. Save the email object into Mongo DB store
  5. Save the email object as .txt/.json file in “uploads” folder
  6. Iterate through attachments if any and store it in “uploads” folder [TODO: MongoDB GridFS]

 

How to run the application?

  1. Clone the repository or download the zip and extract into a folder
  2. set the SMTP configuration in “config.js” – use your email credentials
  3. Set MongoDB configuration – this is optional; if you don’t need it then comment out following line in “mailprocessor.js”
    1. db.save(JSON.stringify(msg));
  4. Set mongo collection as “emails” – This is applicable only if point # 3 is valid.
  5. run “npm update” – this will download all the dependent node_modules recognized from the package.json
  6. run “node mailprocessor.js”

 

Why I created this?

  • Business Motive – We have our home-grown helpdesk / ticketing system for our employees to make any kind of support tickets online however employees are comfortable in sending helpdesk tickets thru email to helpdesk@ourdomain.com; it is our process that the helpdesk / IS team should make a ticket entry for every request received thru email. This application is built to automate ticket creation from email.
  • Passion – I wanted to try out in Node.JS as it is very simple due to the availability of loads of ready made NPMs but choosing the correct npm is the key
  • No full-fledged sample – During the course of development I could not find complete references and samples easily i.e. mailparsing, attachments etc. I collected lot of references from stackoverflow Q&As

Scope for improvement (or TBD)

  1. Email filters and rules to process only those emails which matches a specific condition
  2. Store the attachments in the MongoDB GridFS

Miscellaneous

  •  JavaScript IDEWebMatrix, it is free from Microsoft and simple to use with ability to integrate with Git Repo
  • MongoDB Query Analyser – MongoVUE

Elegant & Re-usable diagrams for Architecture, System Design, Pre-Sales & Proposal activities etc.

If you are a Solution Architect or Presales team members then I am sure you would have prepared numerous diagrams for proposals, system design, Business Process flows etc. using varieties of tools i.e. MS Visio, Excel, Word and online tools like draw.io, lucid chart and many more…

I use all of these above tools but most predominantly Excel for stunning diagrams. Some of my diagrams I made it as excel template for re-usability which you can get it here and modify as per your need.

Sample Diagrams of mine…. [ Inspirations of the diagrams are from internet, reference books and some are my own ideas but I usually create it as fresh] . Download the excel template here.

Application Architecture

image

 

Layered Architecture [MS Patterns & Practices]

 

image

Release Management

image

 

Inter-layer / Component Flow of data [Martin Fowler’s EAA]

image

 

Schedule & Release Plan

image

 

Shift Coverage Matrix

image

Documents Collaboration

TBD

Enterprise Service Bus [Service Agents]

TBD

Stay tuned to check for numerous business process flow [Swim Lane] Visio diagrams soon.

If you get benefit from this template, please drop me an email @ senthilsweb@gmail.com or post it in the comments section which will encourage me to contribute more like this.

Free hosting for 3 years in MS Azure IaaS Platform

Microsoft provides $150 per month worth of hardware for free of cost for 3 years in their cloud infrastructure “Azure” and the program is called “Bizpark” [http://www.microsoft.com/bizspark/azure/]. This is applicable for start-up companies which are less than 5 years old and their annual revenue is less than 1 Million USD.

image

We recommend “Bizpark “ to our SME prospects and clients (Start-ups) to get benefit from the free Scheme. 

Our default deployment architecture we propose in our proposals for start-ups or individuals who would want to built web (REST APIs) application in Microsoft footprint (ASP.NET MVC) to start as small and scale up is depicted below.

DEPLOYMENT ARCHITECTURE

The server infrastructure (hardware & software) will be procured from Microsoft Azure, Infrastructure as service (IaaS). The high level deployment architecture of Microsoft Azure infra is depicted in below

clip_image002

The advantages of Microsoft Azure, IaaS are: (Everyone knows but nevertheless)

  • No upfront Server hardware investment
  • On demand  Server & Storage scale-up or scale-down
  • Load balanced using Traffic Manager (Optional)
  • High availability using Traffic Manager & Geo- redundant (Optional)

HARDWARE DETAILS

No sophisticated hardware is required. The application will be hosted in MS Azure Cloud and on demand the hardware can be scaled up without altering the source code (mainly for fault tolerance and load balancing). Typically we need the following three components

  • Azure Web Site
  • Azure SQL Server Database
  • Azure Cloud Storage

Note: Azure is not only for windows. Azure supports a broad set of operating systems, languages and frameworks— From Windows to Linux, SQL Server to Oracle, .Net to Java, PHP, Python, Ruby, Node.js, Hadoop.

My recommended (IMHO) Books & references for Enterprise Application Architecture (EAA), SOLID Design Principles & Miscellaneous

In the past and now, people close to me, worked with me and working with me keep asking for good reference reading materials (meaning books, articles etc.) for object orientation, design principles, design patterns, Enterprise Application Architecture, Layered Architecture, Loosely Coupled Architecture etc. and how I designed Enterprise Application Architecture in my various client implementations.

Also in contrast to above many a times when I meet my peers who works in the same line of mine, I do talk about what I read & learnt new things in the architecture or what I referred for specific design problems as knowledge sharing. This is like a repeat and redundant occurrences in my day to day life hence thought of sharing some of the excellent evergreen Books, Blog Articles, Video Tutorial sites which Inspired me and helped me to address many design problems I came across.

Some of the materials are highly precious, valuable and eternal wealth like Gold…such references I listed here for others to gain knowledge which I term “My recommended (IMHO) Books & references for Enterprise Application Architecture (EAA), SOLID Design Principles & Miscellaneous

Note: All of these are in C# flavours but technology agnostic (Applies to any programming languages and technologies)

I organized in the following

  • Object Orientation
  • SOLID Principles
  • Loose coupling
  • Dependency Injection
  • Aspect Oriented Programming
  • Domain Driven Design
  • Multi Layer Architecture [Onion Architecture]
  • Miscellaneous

A Twisted Look at Object Oriented Programming in C#

From college days we are are reading about OOP but this one is very good and I liked it very much

https://www.dropbox.com/s/bmugu0e9dk78ybt/A%20Twisted%20Look%20at%20Object%20Oriented%20Programming%20in%20C%23.pdf?dl=0

Pablos SOLID E-book

It is from LostTechies.com. and the author explains SOLID principles with real life use cases and code samples in a simpler manner.

https://www.dropbox.com/s/jn9b3jm8ddhlq7x/pablos_solid_ebook.pdf?dl=0

WroxProfessional.ASP.NET.Design.Patterns.Sep.2010

image

The tile of the book says design patterns but this books mostly about design principles like DDD, DI, ORM, View Model, Request Response View Models, Facade / Services Layer, View Mappers, MVC, Multi Tier & Layered Architecture, Onion Architecture, Cross Cuttings & separation of concerns etc. He explains that how the above principles can be implemented using proven Design Patterns (I think the author felt Design Pattern title is more reachable than Principles for marketing reasons…)

In other words I would say, this books implements Martin Fowler’s  principles from his ever popular book “Patterns of Enterprise Application Architecturein a easily understandable way for any novice coders who wants to venture into Enterprise Architecture or want to be an elegant programmer who can write highly scalable and less friction codes. It is a ALT.NET (Alternate way of doing mainstream dot net…in other ways JAVA / RoR way of doing main stream .Net) book.

31DaysRefactoring

imageThis is also from LostTechies.com and it talks about code refactoring and writing elegant code using proven design patterns and principles

 

 

 

Applying Domain Driven Design and Patterns With Examples in CSharp and .NET

imageThis book is very special to me from my favourite and eminent author Jimmy Nilsson. Chapter 7 & 10 in this book are eye opener for me to learn and implement Rules Engine & Aspect Oriented Programming (AOP) in specific to the design & architecture I involved in.

 

The Spring.NET Framework

We  heavily depend on using Spring Framework for all of our development in MS.NET & JAVA. Though it has various modules but we depend on the following

  • Dependency Injection
  • Aspect Oriented Programming
  • Web/RESTful Service exporting from Plain Old C# Objects (POCO) i.e. Convert the Facade / Service Layer as .asmx / Restful Webservices
  • Expression Evaluation
  • Quartz Scheduling
  • nVelocity Templating (Recently we migrated to Razor Templating engine)
  • RabbitMQ Client library

I strongly strongly recommend anyone to read the Spring documentation.

Omar Al Zabir’s “Build Truly RESTful API and Website using Same ASP.NET MVC Code”

He blogged about how to write clean and neat RESTful Services especially the route naming conventions.

As the title says, he also talks about how to use the Controllers, the C in the MVC as Restful API (Specific to ASP.NET MVC) instead of writing an additional API layer i.e. ASP.NET Web API.

I recommend the the Route Naming Conventions for any technologist but the Controller to RESTful API is for my team members and any ASP.NET developers.

Programming asp.net mvc 4

image

As the name says, It is not about OOP or Design principles but about ASP.NET. Though many books are available for ASP.NET but this one is excellent and very practical for anyone who wants to build enterprise class web application. It is written in such a way that the followers will not miss anything in the website development i.e. Infrastructure, ORM, DI, Security, Caching, Deployment etc..

Pluralsight Online Video Tutorials (Paid Tutorial)

image

Worth paying USD $15 per month for pluralsight. It has everything and releasing new stuffs to keep us up-to-date in the technology and other soft skills.

My recommendation to all who would want to use pluralsight is; follow this tutorial generously on regular basis i.e. every day for 30 mins. to 1 hour for best results.

I usually watch every day before go to sleep, during weekends and leisure time through my iPad. I often follow the author coding (pause & stop the video) for my hands on and real life client projects.

Light bulbSome of the books I mentioned here are available for free to download from http://it-ebooks.info/tag/programming/ 

Finally……….

Being an Architect, I read a lot like any others in our profession and look for opportunity to implement what I learnt in theory, yes I am hunger for new things, proven way of solving problems, look for simple solutions, want to write elegant and maintainable codes, never shy in continuous refactoring etc. in order to do so, I must equip hence I read and refer many different programming languages,frameworks & technologies than the one I mostly into (C#, Object Oriented JavaScript, Node.JS, Ruby on Rails, Mobility) and how they address same problems differently.  Usually while reading for first time (or second, third) ,  I will (or may) not understand all of it which I simply skip and keep going the rest which I feel helped me a lot. Also during my go though I never forget to bookmark important references.

Vmoksha’s Mobile Enterprise Application Platform (MEAP) Architecture

A well defined Architecture is critical for any development. Having architecture in place, we can implement technical governance across the team in terms of coding style, standards etc. and also this will help to refactor the code in a centralized place on a continuous basis a.k.a continuous re-factorization.

Having said that, we developed our our MEAP architecture and all our mobile application development honours this. This is the tweaked version of our Web application development architecture, the major difference lie in the tools & technologies used for developing the UI layer. In a nutshell most of the mobile development follows similar pattern i.e. RESTful Architecture using JSON(P) is the primary content type to transfer the payload (data) between client and server. This architecture is Technology Agnostic, predominantly we use the following technologies to develop our Server Components i.e. Back end layers (Facade, Data Access, RESTful Services)

    • .NET (WCF, NancyFx)
    • Ruby On Rails
    • PHP

 

Architecture

MEAP

Flow of dataFLOW-OF-DATATechnology BackboneTechnology Platform