Why J2EE Projects Fail
My suspicion is that this will go down as the most valuable session I attended here. Rod Johnson, creator of the not-quite-mainstream but wildly popular Spring framework, presented this session based on his observations consulting on J2EE projects.
He introduced the discussion by describing the challenges of enterprise development. It's hard to build: integrating disparate systems, performance problems, and complex domain models. It's hard to test and maintain: software is NEVER finished, and timelines are often too compressed for adequate test scripting/planning.
He then laid out several reasons for project failure, both general and unique to J2EE technologies.
Failure to understand requirements
Rod suggested a simple checklist for applications:
Ideology
Rod offered a historical view of the ideology of early J2EE architects and developers. There was, in his words, an elitism that Java is the way, the truth, the light, and that RDMBS is vastly inferior. He also mentioned the proliferation of remote method invocation in early J2EE applications beyond what may have actually been necessary.
The overriding principle throughout all of this was that the past ideology of J2EE has been one where the technology was more important than the problem.
Rod talked a lot about the need to view the entire technology stack as potential tools for solution design, and not to just think of Java. As he said, there is no such thing as an enterprise Java developer... only enterprise developers. Java has a role, but so do other technologies.
Lack of attention to performance
Poor performance appears to be the most frequent cause of J2EE project failure. Performance always comes up in relation to EJB and RMI, and he reinforced that here today.
Rod talked about the dangers of a "first get it to work, then optimize" development methodology. He strongly encouraged vertically sliced proof of concept development early on to validate architectural decisions. Because changing architectural decisions is a costly exercise, it is important to get these decisions right early in the process.
Loosely coupled application layers are preferred, because they provide independence and flexibility from cascading impact of architectural refactoring.
Team dynamics
One of the issues in team dynamics Rod spent the most time talking about was the idea of the God-like architect. According to him, there are two types: the non-coding variety who consider UML a valid prototype, and the "prescriptive framework builder" who essentially micromanages developer decisions at every level of the project. Communication and mutual respect are key for architects and developers to happily collaborate.
Having too big a team can also cause problems. It tends to lead to treating symptoms rather than reducing complexity. In other words, he's saying throwing more and more developers on a project to meet a deadline, this is probably an indication that some other problem is at work in the underlying architecture of the solution. He used several military analogies to drive home the point of leverage... It's not about numbers, it's about leverage.
Frameworks and design patterns provide the leverage we need to be successful. So if we're throwing developers onto a problem, this is an indication that the project is not adequately leveraged against reusable technologies and proper architectural planning. We don't see a lot of this at NI, he was mostly talking to fellow consultants in the room, but still the thought of the importance of leverage is perfectly valid for NI.
Too much code
Another side effect of large teams is simply that they tend to generate much more code. One of his objections to the complexity of J2EE and EJB at present is that it is very difficult to find actual business logic amidst all the required configuration code.
Too much code is also a reflection that the architecture does not adequately leverage third party frameworks, or perhaps is excessively using EJB. Either way, J2EE projects typically tend to result in lots and lots of code that has to be maintained for the life of the production application.
Reinventing the wheel
Lots of in-house development is wasted on services already provided by the container (connection pools, persistence, logging, etc). If that isn't bad enough, the rest of the services are already provided by third party frameworks like Struts or Spring.
Rod stressed the importance of not building an in-house methodology from scratch. I'm not sure what he would have to say about our almost complete lack of a methodology at all. He did say that one of the most important trends he had seen in industry over the past 18 months or so was the trend toward third party frameworks.
Persistence issues
He very directly said that managing your own custom JDBC code is a bad idea, unwise and ineffective. The code is verbose due to the requisite exception handling, a problem compounded by the risks associated with such error-prone development. As he put it, "JDBC is not a good API for developers to use."
Object-Relational Mapping (ORM) tools can both help and hurt your situation. They can reduce development time but create performance issues. They can greatly simplify use of a complex domain model at the object level, but ORM can also add complexity to what otherwise would have been simple problems. So it is incumbent on the developer to do due dilligence on whether ORM makes sense for your use cases. However, he did say at a minimum to avoid raw JDBC.
Performance failure
Performance is not part of architecture, which reinforces the need for early vertical slices for design validation. It can come from excessively layering remote calls or abusing ORM tools for services or operations that could much easier have been performed on the database.
Lack of testing
He lamented the ad hoc nature of most testing for server side Java applications. He suggested that testing feature as much automation as is possible.
Recommendations
He wrapped up by saying that you should never trust an architecture until you've seen a proof-of-concept that aligns with your needs. He suggests establishing clear metrics on performance, level of effort, and maintainability. And finally, he encourages us to automate everything (test) that we can to improve efficiency and reliability.
The best line of the session he gave? "We need to progress to higher levels of abstraction." That's exactly the problem I think we have at NI.
He introduced the discussion by describing the challenges of enterprise development. It's hard to build: integrating disparate systems, performance problems, and complex domain models. It's hard to test and maintain: software is NEVER finished, and timelines are often too compressed for adequate test scripting/planning.
He then laid out several reasons for project failure, both general and unique to J2EE technologies.
- Failure to understand or communicate requirements
- Ideology
- Lack of attention to performance
- Bad team dynamics { God-like architect, team too large, overworked resources }
- Lack of appropriate testing strategies
- Inability to meet performance goals
- Too much code
- Assuming the very small translates to the very big
- Reinventing the wheel
- Persistence issues
Failure to understand requirements
Rod suggested a simple checklist for applications:
- What should it do?
- How should it do it?
- Whether it actually does it
Ideology
Rod offered a historical view of the ideology of early J2EE architects and developers. There was, in his words, an elitism that Java is the way, the truth, the light, and that RDMBS is vastly inferior. He also mentioned the proliferation of remote method invocation in early J2EE applications beyond what may have actually been necessary.
The overriding principle throughout all of this was that the past ideology of J2EE has been one where the technology was more important than the problem.
Rod talked a lot about the need to view the entire technology stack as potential tools for solution design, and not to just think of Java. As he said, there is no such thing as an enterprise Java developer... only enterprise developers. Java has a role, but so do other technologies.
Lack of attention to performance
Poor performance appears to be the most frequent cause of J2EE project failure. Performance always comes up in relation to EJB and RMI, and he reinforced that here today.
Rod talked about the dangers of a "first get it to work, then optimize" development methodology. He strongly encouraged vertically sliced proof of concept development early on to validate architectural decisions. Because changing architectural decisions is a costly exercise, it is important to get these decisions right early in the process.
Loosely coupled application layers are preferred, because they provide independence and flexibility from cascading impact of architectural refactoring.
Team dynamics
One of the issues in team dynamics Rod spent the most time talking about was the idea of the God-like architect. According to him, there are two types: the non-coding variety who consider UML a valid prototype, and the "prescriptive framework builder" who essentially micromanages developer decisions at every level of the project. Communication and mutual respect are key for architects and developers to happily collaborate.
Having too big a team can also cause problems. It tends to lead to treating symptoms rather than reducing complexity. In other words, he's saying throwing more and more developers on a project to meet a deadline, this is probably an indication that some other problem is at work in the underlying architecture of the solution. He used several military analogies to drive home the point of leverage... It's not about numbers, it's about leverage.
Frameworks and design patterns provide the leverage we need to be successful. So if we're throwing developers onto a problem, this is an indication that the project is not adequately leveraged against reusable technologies and proper architectural planning. We don't see a lot of this at NI, he was mostly talking to fellow consultants in the room, but still the thought of the importance of leverage is perfectly valid for NI.
Too much code
Another side effect of large teams is simply that they tend to generate much more code. One of his objections to the complexity of J2EE and EJB at present is that it is very difficult to find actual business logic amidst all the required configuration code.
Too much code is also a reflection that the architecture does not adequately leverage third party frameworks, or perhaps is excessively using EJB. Either way, J2EE projects typically tend to result in lots and lots of code that has to be maintained for the life of the production application.
Reinventing the wheel
Lots of in-house development is wasted on services already provided by the container (connection pools, persistence, logging, etc). If that isn't bad enough, the rest of the services are already provided by third party frameworks like Struts or Spring.
Rod stressed the importance of not building an in-house methodology from scratch. I'm not sure what he would have to say about our almost complete lack of a methodology at all. He did say that one of the most important trends he had seen in industry over the past 18 months or so was the trend toward third party frameworks.
Persistence issues
He very directly said that managing your own custom JDBC code is a bad idea, unwise and ineffective. The code is verbose due to the requisite exception handling, a problem compounded by the risks associated with such error-prone development. As he put it, "JDBC is not a good API for developers to use."
Object-Relational Mapping (ORM) tools can both help and hurt your situation. They can reduce development time but create performance issues. They can greatly simplify use of a complex domain model at the object level, but ORM can also add complexity to what otherwise would have been simple problems. So it is incumbent on the developer to do due dilligence on whether ORM makes sense for your use cases. However, he did say at a minimum to avoid raw JDBC.
Performance failure
Performance is not part of architecture, which reinforces the need for early vertical slices for design validation. It can come from excessively layering remote calls or abusing ORM tools for services or operations that could much easier have been performed on the database.
Lack of testing
He lamented the ad hoc nature of most testing for server side Java applications. He suggested that testing feature as much automation as is possible.
Recommendations
He wrapped up by saying that you should never trust an architecture until you've seen a proof-of-concept that aligns with your needs. He suggests establishing clear metrics on performance, level of effort, and maintainability. And finally, he encourages us to automate everything (test) that we can to improve efficiency and reliability.
The best line of the session he gave? "We need to progress to higher levels of abstraction." That's exactly the problem I think we have at NI.
posted by Anonymous at 4:33 PM
0 Comments:
Post a Comment
<< Home