Artificial Intelligence
Consulting
Development ServicesEnterprise systems are becoming more complex and robust, heterogeneous and solid-state. Big data volumes grow exponentially, but how ready are software developers to face such a reality? Aren't many developers today wasting their time chasing after new innovative cool frameworks instead of working on resolution of critical challenges in enterprise app development? Isn't this whole situation getting worse year after year?
In this article we'll review the main challenges facing enterprise app development these days and how they can be solved.
Why is enterprise app dev a trending topic today?
Some of the main reasons are:
- Enterprise apps have rather complex architectures and complicated back-end integrations;
- They should be able to work with big data efficiently;
- Distributed development environment;
- They're critical for business sustainability;
- Cost of mistake in enterprise app dev is very high!
In most cases, enterprise app developers are pressured by time constraints, compatibility issues and scalability, and they rarely have the opportunity to experiment with unfamiliar tools and out-of-the-box approaches.
From enterprise viewpoint, small projects (up to a year of man-hours) are a testing site for a new technology or framework, while the true enterprise potential can only be realized in hardcore long-term projects. There's a strong mutual agreement within the industry that each new enterprise development technology should go ripe and mature and it can take up to 5 years for this to happen. If something has stayed for 5 years and hasn't died out, it's worth applying to enterprise app dev! Such an approach is actually detrimental for this type of app development and enterprise tech evolution.
Enterprise development issues and challenges
Complexity
It is one of the most serious issues in enterprise app dev today. Enterprise infrastructure and ecosystem are complex in nature and so is application development. Not to be unfounded, but the average size of one of the enterprise project modules within a single application is 300-500 mb of source code. And such modules are many! It's just impossible to take all this information and put it into someone's head at once: no matter how hard you try, some of that information will just not fit in and will fall out. Because it's too much of information!
The problem is that many enterprise apps that exist today are not taken care of properly. Each module lives its own life, is usually controlled by a certain department within organization; developers of different modules don't communicate with each other, don't share knowledge and don't coordinate solutions they implement. As a result, no one within the enterprise can see the picture as a whole and nobody is on the same page, nor do they have any shared vision about how to evolve the entire application. Developers of different modules solve the same or similar issues differently which is a bummer.
Also, you can't analyze the system properly because as you analyze one module, the other one has changed and no one has informed you about this while you were checking another module. It looks like a vicious circle that never ends.
Monolithic nature of apps
Because many enterprise apps are monolithic, one part of the system can't be viewed separately from the rest of the system, i.e. you can't take one module and just replace it with a newer and better one or re-write it from scratch. It already contains embedded integrations with dozens of adjacent modules and other dependencies from dozens of other modules you may not even notice in a timely manner. One day, all of a sudden, you'll find out one of the integrations malfunctions and it's going to be a real pain in the neck!
Heterogeneity
That's another huge issue as far as enterprise systems are concerned. By heterogeneity I mean a havoc of servers, technologies, programming languages, frameworks, libraries and development approaches mixed together. Even when the same software is used, different modules usually use different versions of that very software. Blended with monolithic nature and overall complexity, enterprise app's heterogeneity prevents the system from scaling and evolving any further.
Unfortunately, many enterprise development teams today boast having a multitude of tools and platforms they use, which isn't always good.
External integrations
Not only are enterprise apps complex and heterogeneous, not only are they impossible to analyze piece after piece, they also don't integrate well with other similar monolithic and messy apps. Did you know that in the USA, which is ahead of the curve as far as technology innovations, all TV broadcasting companies still synch their programs by downloading files via FTP? Yep, archived 4 GB XML! What do you think about such integration? It proves that enterprise apps are supposed to be built once and forever, doesn't it?
Big Data
Despite the above issues, many enterprise apps function more or less OK and fulfill their goals more or less properly. But big data is really a serious challenge! As you can see in the picture below, the volume of data stored in enterprise systems grows exponentially and will earlier or later exceed the current bandwidth and capabilities of enterprise solutions and relational databases.
Source: IDC Digital Universe, 2012
How is this data volume created? Firstly, almost all data that gets into an enterprise system stays there forever. Secondly, every day business creates new ways to obtain new data. And all this data should be stored somewhere and wait for Apocalypse to happen.
If current data storage paradigm ever changes, this will lead to malfunction of thousands of solutions accumulated over a very long period of time, many lacking any type of documentation or even a source code.
As mentioned above, the average size of one enterprise app's module is 300-500 mb of source code. This is the module that took 10 years and 3-5 developers to build. One enterprise app can have from 50 to 1,000 different modules and weigh dozens of terabyte of source code. It's unimaginable how one can take and re-write all this. That's not something that can be refactored within a particular timeframe. This requires a more in-depth understanding and non-trivial solutions.
Legacy code
Of course, most of enterprise apps are hardwired to legacy code. This happens because you can't build an enterprise application fast; it usually takes years or decades to create. Any software developer wishes they could throw legacy code away and write a new one from scratch. As they say, your system has one huge flaw - it wasn't written by you. But because it's impossible to re-write the whole enterprise system with a clean slate (as the company will fall apart right away and all of its operations will be paralyzed), you either have to retain a status quo or deploy a new system while retaining the old one.
Would you like to learn more
about talent solutions
How these issues can and should be tackled
Once they bump into an issue, many software developers are looking for a silver bullet to solve it. It can be a new, just released framework or a new programming language or technology. This approach increases heterogeneity. Many team leads and PMs are well aware about this habit of developers (at the end of the day, tech leads and PMs often originate from developers and testers), and they don't normally encourage use of unfamiliar tools. This usually drives developers crazy and makes them think they work with silly management. But PMs and team leads also have a favorite silver bullet which is a development methodology. They're obsessed with those methodologies as much as some developers are obsessed with new frameworks. And they love applying those methodologies to each and every project no matter how relevant they are. So, it's another vicious circle that is so hard to break through.
Some of the possible ways of solving the issues of complexity, heterogeneity and big data in enterprise apps include, but aren't limited to:
-
Communication should be more important than technologies
Many experienced project managers agree that if there's a development issue on the project, it's rarely a technology issue, but rather an issue of interaction between team members (disintegrated teams). The problem usually underlies in data transmission and processing. According to Pareto principle, 80 percent of results come from 20 percent of efforts. Prior to trying to solve issues in technological ways, you need to align all teams involved in modules development for the enterprise app and put them on the same page, encourage knowledge sharing and foster shared vision of the application being developed. DevOps and Agile practices can definitely help improve communication between developers and set the ground for better mutual understanding in the future.
-
Architecture solutions should be more important than tools
Making a decision about what tech stack will be used for enterprise application development PRIOR to making any architecture decisions is a real plague in modern software engineering. If we review presentation titles at most of IT conferences today, we'll see topics like new features of framework X, use of language Y in certain systems or a new API from library Z. But we won't likely see anything relating to complex cases of enterprise app's decomposition or determination of a suitable technology for boxed architecture solutions.
Today, technology stack dictates what type of architecture will be built. To many enterprise developers, it's a way to the blind alley. In best case scenario, it's the architecture that should affect your choice of stack, not vice versa!
-
Standard solutions should be more important than perfect compliance of tools and project tasks
In any difficult situation you should better choose solution that's already being used in the system. As the saying goes, the best way is one that driver knows well. The more standardized your solution, the easier it'll be to support the application and the more flexibility you'll have shuffling developers from one project to another. The latter will help you reduce app's heterogeneity and make it less monolithic.
To achieve this, it's highly recommended that you develop and launch your project specific system of standards and make it a mandatory rule for all organization's departments to abide. For instance, if you use SOAP for all app's modules integration, keep using SOAP for a new integration even if there're faster and easier ways to do so. Also, create a single hub of micro services and make sure each micro service has its own documentation and description of its interface and functional features. Each PM who adds a new service to the hub should be accountable for providing this information. This will help eliminate the situation when new functionality is built from scratch, and increase use of reusable components and ready services to speed up development and optimize budget.
I know that many programmers will start whining about having to write all that information, but they'll stop doing it once they're used to it as a regular routine.
When it comes to big data, the following issues come in mind:
- When we add a robust big data solution to a large and complex enterprise application, we inevitably increase app's heterogeneity;
- When we add a new massive dataset, we inevitably increase app's complexity
Big data is very important, but how not to misuse it should be a big concern for enterprise developers! Developers should stop looking for a silver bullet everywhere and should use their common sense instead.
For example, let's take a look at NoSQL that is often deemed as a silver bullet by many developers, especially those with little professional experience. Let me remind you of the following NoSQL issues. As you may know, any database can provide 2 of 3 features: speed of access, consistency and distributiveness. While relational databases provide the first two from the box, non-relational databases provide the first and the third features, i.e. speed of access and distributiveness. When it comes to enterprise development (where cost of transaction can reach several million dollars), no box consistency means the end of the world.
In order to use big data effectively for enterprise app development, you can use the following recommendations:
- Don't use big data for every single app's module; instead, use big data for those modules only that badly require big data to work successfully. That being said, create a list of data that you'll apply your big data tools to. This part of the system should be well encapsulated and protected from the rest of the system by a properly written integration interface. This will prevent other parts of the system from adding their data into your data hub or using methods not described in / permitted by the interface.
- Create your app specific data aging principle to avoid storing obsolete terabytes of unclean data in database.
To conclude with, today's enterprise application development looks more like a feudal system where each piece of land has its own vassal serving a certain king. It's complex, monolithic and heterogeneous, with every stakeholder being on their own page. Some of the ways to tackle these and other issues are to use standard solutions for all same-type issues, implement DevOps and make sure enterprise app's architecture dictates your choice of technologies, not vice versa. And what's your take on this?
Would you like to learn more
about talent solutions
