Have all requirements been defined and are they current?

How often are stakeholders provided with a complete project status (i.e. includes concurrent updates on scope, schedule, resources and risk) and is the status report a two way communication vehicle?

Have change requests accounted for more than a 10% change to the original schedule?

When did the team start developing the verification plan?

Is the project on schedule as measured by the deliverables committed to in the original schedule?

Do team members know if their tasks are on the critical path?

Are all hours being recorded, and if so are the actual hours and the estimated hours within + or – 10% at the task level?

Has the introduction of new technology accounted for more than a 10% improvement in schedule over previous “similar” projects?

Can the team members show you a project dictionary where all technical and user terms are defined?

What are the top three risks associated with the project?

Have all requirements been defined and are they current?

The majority of projects that fail to meet expectations do so because the project team does not understand stakeholder needs.  Even when the team captures the requirements at the beginning of the project, they can still miss the fact that stakeholders and their needs will change over the course of a project, especially strategic projects that take more than 6 months to complete.  If a project is 20% complete and there are still unresolved requirements, it is an indicator of a project that needs a more in depth assessment.

How often are stakeholders provided with a complete project status (i.e. includes concurrent updates on scope, schedule, resources and risk) and is the status report a two way communication vehicle?

A status does not provide a complete picture unless it is looking at all of the major attributes as of the same point in time.  Therefore a status should contain timely information on scope, schedule and cost (resources).  By definition Earned Value reporting does this for you, however if the data in the Earned Value report is not timely then status can be misleading.  If the information is relatively current (days) then it is a great and timely tool.  If the data is weeks or even months old, then the accuracy can be challenged – remember a lot can happen in 6 weeks.

The second part of this question is a subtle reminder that requirements can and do change – the status report is a good forum for gathering information on whether the requirements are evolving.

Have change requests accounted for more than a 10% change to the original schedule?

This is another question trying to assess whether requirements were really understood.  The 10% number is arbitrary – maybe it should be 5% or maybe it should be 15%.  The evaluation is used along with the rests of the data to determine if the team understood the objectives of the stakeholders.   A high volume of change request, especially those that are generated as a result of viewing early prototypes or models is typically an indication that the development team, users, and stakeholders were not in agreement on the requirements.

When did the team start developing the verification plan?

Verification planning should start with the development of the requirements.  If the team has no means of verifying the deliverable being produced meets the requirement, then how do they know it was a valid or even attainable requirement?

Is the project on schedule as measured by the deliverables committed to in the original schedule?

This is a simple, but loaded question with two operative phrases; deliverables and original schedule.  We’ve seen too many projects that are on track in terms of hours spent, however they have failed to produce a deliverable, so how do they or you know the project is on track?  The second part of the question is evaluating whether the team ever created a baseline.  It may seem hard to believe, but we have seen project teams revise the schedule right before a status meeting and then announce to the world they are on schedule, only to miss the year end objective by months.

Do team members know if their tasks are on the critical path?

Critical path defines the schedule.  Every PM will know the critical path; however a sign of a well-functioning team is their ability to communicate.  So not only should the PM know the critical path, but all team members, especially those working the tasks should know it as well.

Are all hours being recorded, and if so are the actual hours and the estimated hours within + or – 10% at the task level?

Estimating is quite simply guessing.  There are several potential issues with a project that is trending over or under 10%.  (BTW a negative run rate can be just as detrimental to project success as an overage.)  If the answers indicate the range is being exceeded, you will need to ask a lot more questions to identify route cause of the difference between the plan and actual. Do resources have the right skill level as predicted when developing the estimate?  Was the estimate bad? Was the work poorly defined? Are people charging overhead time (like time spent preparing a status report) to tasks because they have no place to charge overhead time? Are people working on the project part time whereas the plan assumed full time staff?  

Has the introduction of new technology accounted for more than a 10% improvement in schedule over previous “similar” projects?

This is an indicator of false hope.  Yes, new technology will improve productivity (over time) however it is rare for the introduction of new technology to produce an order of magnitude change the first time it is used.

Can the team members show you a project dictionary where all technical and user terms are defined?

At the end of the day, everyone on a project needs to be speaking the same language.  The best way to demonstrate that is by having a project dictionary.  The same word has a very different meaning depending on your background. For example when you ask someone the temperature are they answering in Fahrenheit (USA) or Celsius (Europe)? There is a huge difference between 40 degrees F (relatively cool) and 40 degrees C (scorching).  A document (project dictionary or even a project wiki site) will help get everyone on the same page.

What are the top three risks associated with the project?

There is no “right” answer to this question except that the PM and all of the team members should have an answer right off the top of their head.  If you ask and they tell you they need to look it up and get back to you – you have a project that is challenged. If they (PM and team members) all list the same three risks, then you have a team that communicates very well, a critical attribute to team success.



In his June 2013 newsletter PD Mentors, Ken Mosteller quoted the results of his own survey, in which he asked which management paradigms are a trend or a fad.  Apparently, 100% of respondents judged Lean Systems Engineering (Lean SE) to be a fad[1].  In our subsequent personal conversation Ken stated that indeed the survey was quite unscientific.  I accept Ken's argument that unscientific surveys are still useful as initiators of a public discourse, and I offer this note in that spirit.

I am not surprised by the survey: we meet this reaction from newcomers into Lean SE frequently. Systems Engineers have been fighting for years for increased resources in PD programs, often against overwhelming forces, and are naturally suspicious of Lean which sounds like "less Systems Engineering".  In fact, Lean SE requires more and vastly better Systems Engineering and Program Management than traditional, in order to uncover huge productivity reserves in our programs, and vastly reduce various categories of program waste, such as waiting, defects and rework, excessive refinement, and others.  Formal measurements indicate that in large performing programs, on average 88% of charged program time is spent on non-value adding activities, and 82% of requirements need to be changed.  These are huge productivity reserves that Lean for Systems Engineering helps to identify and utilize for the benefit of the program. 

Lean SE does not take eliminate any value-adding activity and does not cut corners.  Quite the opposite: it represents a large spectrum of best practices in complex technology environments, with a general focus to improve program value and customer and other stakeholders satisfaction; and reduce waste, delays, costs, as well as frustrations and "firefighting".  Lean SE yields programs which are more stable, predictable and robust.  Lean SE promotes the integration of Systems Engineering and Program Management to reduce unhealthy friction.  Lean SE strengthens the culture of trust, openness, respect, empowerment, teamwork, coordination and communication, and drive for excellence.  Lean SE practices encourage healthy relationships between all stakeholders; better coordination between parties handling any complex transaction; streamline work flow while promoting robustness and right the first time; and identify best methodologies for complex system design. They place emphasis on good preparations, planning, frontloading, and preventive measures.  They emphasize process optimization, standardization, continuous improvement, and long - term thinking.  They describe best practices for human resource management; creation of communities of practice and knowledge databases; capture of lessons learned; continuous education and learning.   

The Lean Enablers for Systems Engineering[2], and the subsequent Lean Enablers for Managing Engineering Programs[3] [free download: http://hdl.handle.net/1721.1/70495] have been developed by teams of 20 global experts in Systems Engineering, Program Management and Lean, supported by advisory groups totaling 300 practitioners, validated by numerous surveys and benchmarking with past programs and with NASA and GAO recommendations.  The Enablers have been developed to respond to 10 following program challenges:  1) Firefighting - Reactive Program Execution; 2) Unstable, Unclear and Incomplete Requirements, 3) Insufficient Alignment and Coordination of the Extended Enterprise, 4) Locally Optimized Processes that are not Integrated Across the Entire Enterprise, 5) Unclear Roles, Responsibilities, and Accountability, 6) Mismanagement of Program Culture, Team Competency, and Knowledge, 7) Insufficient Program Planning, 8) Improper Metrics, Metric Systems, and KPIs, 9) Lack of Proactive Program Risk Management, 10) Poor Program Acquisition and Contracting Practices.  Don't we all identify on a daily basis with these frustrations in our programs?  The Lean Enablers provide actionable and practical practices on how to reduce these frustrations. 

The two mentioned books have been praised and endorsed by top practitioners, academic experts, program managers, as well as PMI President and INCOSE President.  Collectively the two mentioned books received two Shingo Awards, INCOSE Best Product Award, and three smaller INCOSE awards.

The enablers are already practiced by a number of companies with amazing success, saving 20-40% of program effort with only a few enablers implemented at a time.  Our web page[4] contains specifics. 

To summarize, Lean SE is not a fad.  It is the state of the art in Management of Engineering Programs. Any organization, which ignores this established knowledge risks irrelevance.  And Lean SE makes our professional life more pleasant and rewarding.      





I was recently asked if I could explain the difference in the value proposition between obtaining a Certificate in Systems Engineering offered by a local university versus becoming a Certified Systems Engineering Professional.  I realized that the answer may be of value to many people, not just to the person who requested the information.

Let’s start the comparison by looking at INCOSE’s CSEP.  The CSEP is based on a document (INCOSE Systems Engineering handbook) that was developed by a world-wide organization, and which has been recognized as a technical reference by another world-wide organization, ISO.  To qualify for a CSEP, one must have an undergraduate degree in a scientific field, a minimum of 5 years system engineering experience across multiple disciplines, have references that also have system engineering experience, and one must pass a rather difficult test based on the aforementioned internationally recognized system engineering handbook.

Now let’s look at the criteria for obtaining a certificate.  The definition of a Certificate is not standardized within the USA and the value of the certificate can vary greatly between colleges.  One program that I am aware of is offered through the “School of Continuing Education”.  Basically, if you paid your fees and attended the courses, this college would issue you a certificate on their letterhead.  The value of the certificate was in direct proportion to the value of the college’s name printed on the certificate; however, none of the courses could be transferred into any type of graduate school program.

On the other hand, another certificate program required the student to have qualified for and matriculated into the schools’ graduate school of engineering.  These courses either had tests associated with them, or submission of papers which would be graded by the instructor. Upon completion of the first 4 courses, the student could stop and obtain a certificate.  If they wanted to then obtain a Masters in Systems Engineering, these courses would qualify at not just this college, but any recognized graduate program.

So what is the difference between a CSEP and a Certificate in SE?  Unfortunately there is no clear cut answer.  The constant is the CSEP; the variable is the Certificate.  If you are going to enroll in a Certificate Program instead of obtaining the CSEP, then pay close attention to who is issuing the certificate.  Is it being offered and taught by the faculty of an existing department within the college, or is it being offered as a means to leverage the fixed space on the college campus? 

Personal opinion – unless you are planning to eventually obtain an advanced degree in engineering, I’d elect to get the CSEP. 

Final Note – not all certifications are as rigorous and or as well recognized as INCOSE’s CSEP.  I was once offered a chance to become a certified instructor.  All I had to do was submit a video tape (this was pre- digital recording days) and this organization would have their experts (none of whom were named) review the tape and send me my certificate.  No test, no experience, no personal observation, no references required.  (By the way – they also wanted a couple hundred dollars in fees.)