Cell Phone Software Forum
| View previous topic :: View next topic |
| Author |
Message |
David Lightstone
Guest
|
 Posted: Tue May 29, 2007 6:11 pm Post subject: Re: Testing in large scale projects |
 |
|
"Vladimir Trushkin" <Vladimir.Trushkin@gmail.com> wrote in message
news:1180444867.730004.40760@q66g2000hsg.googlegroups.com...
| Quote: |
On May 24, 4:59 pm, "David Lightstone"
david._NoSpamlightst...@prodigy.net> wrote:
How should the development effort be structured so that the effort can be
tracked (progress)? i.e. mechanisms to both organize the material in need
of
being tested and track development progress.
I think this is a reasonable desire to have all tests referencing to
requirements. At the very least there should be at least one test
checking whether or not a requirement is implemented correctly. I am
not sure about unit tests. I would by no mean try covering system
level requirements with developer tests because if requirements are
written in a good way and describing the business logic and unit tests
are aimed at testing paths through the code I see a disconnect here.
So, I would recommend covering system level requirements with system
level tests and designing requirements with unit tests.
Now to the point. The best way of tracking development of a complex
system is breaking it down into several parts. Each iteration shall
include full scale of testing, starting at unit level, then advancing
to integration testing and finally system testing. This approach
implies that you know when and which requirements will be ready for
testing. Some additional time shall be allocated at the backend of the
project to fit in integration and system testing for the complete
system.
|
The key here is " implies that you know when and which requirements will be
ready for testing."
I would like to believe that this knowledge is available, but experience
tells my otherwise. That is on a death mzarch (ie organization understaffed
and having an agressive schedule such tracking quickly goes degrades.
| Quote: |
Assume Doors as a requirement management tool. Set up any sort of tracing
mechanism that you want.
Easy. Requirements have numbers. Test names have the requirement
number encoded, or if you have a system like TestDirector it's even
simpler, just import your reqs into TestDirector and reference your
tests to achieve the coverage goals.
Subquestion - when you are done do you have verification of validation of
the requirements?
Verification of validation? Not quite get it. But if you mean whether
or not we look at the final results of requirements coverage with test
then yes, we do it.
-----
Best Wishes,
Vladimir
|
|
|
| Back to top |
|
 |
| |
Ads |
Advertising
Sponsor
|
|
Vladimir Trushkin
Guest
|
| Posted: Wed May 30, 2007 3:04 pm Post subject: Re: Testing in large scale projects |
|
|
On May 29, 5:11 pm, "David Lightstone"
<david._NoSpamlightst...@prodigy.net> wrote:
| Quote: |
The key here is " implies that you know when and which requirements will be
ready for testing."
I would like to believe that this knowledge is available, but experience
tells my otherwise. That is on a death mzarch (ie organization understaffed
and having an agressive schedule such tracking quickly goes degrades.
|
If it's as bad then without changing the environment you will hardly
be able to succeed.
I strongly believe that in a large scale project there must be some
assumptions about what you are going to implement, how, and what it is
going to take. Based on this information one can do some planning, in
which features can be grouped into several integration builds.
Otherwise I would not give more than 1% that the project will be
completed successfully and on time.
The goal of iterative development is keeping a small part of the
biggest picture in a close sight, one at a time. People cannot
effectively work with a large amount of objects. Even having a close
look there is a chance to miss the target. But the smaller the step,
the less chance is out there to miss the target so badly that there
will be no possibility to get back on track. "Divide and conquer" in
some sense.
----
Best Wishes,
Vladimir |
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
David Lightstone
Guest
|
| Posted: Wed May 30, 2007 5:37 pm Post subject: Re: Testing in large scale projects |
|
|
"Vladimir Trushkin" <Vladimir.Trushkin@gmail.com> wrote in message
news:1180523080.479652.279790@u30g2000hsc.googlegroups.com...
| Quote: |
On May 29, 5:11 pm, "David Lightstone"
david._NoSpamlightst...@prodigy.net> wrote:
The key here is " implies that you know when and which requirements will
be
ready for testing."
I would like to believe that this knowledge is available, but experience
tells my otherwise. That is on a death mzarch (ie organization
understaffed
and having an agressive schedule such tracking quickly goes degrades.
If it's as bad then without changing the environment you will hardly
be able to succeed.
I strongly believe that in a large scale project there must be some
assumptions about what you are going to implement, how, and what it is
going to take. Based on this information one can do some planning, in
which features can be grouped into several integration builds.
Otherwise I would not give more than 1% that the project will be
completed successfully and on time.
|
Such assumptions are present, but when the deadlines approach risk
mitigation is activated (often independently developed on the fly). These
envariably entail change to the scope of the deliverable, change to the
scope of the testing that will be performed, etc. The various risk
mitigation plans need not necessiarly be consistent or coordinated. Ergo you
don't know what your will be working against. Although described as
interative (initially) it degenrates to diffusive. That is once thrown over
the wall, the recipeint gets to characterize and adapt
| Quote: |
The goal of iterative development is keeping a small part of the
biggest picture in a close sight, one at a time. People cannot
effectively work with a large amount of objects. Even having a close
look there is a chance to miss the target. But the smaller the step,
the less chance is out there to miss the target so badly that there
will be no possibility to get back on track. "Divide and conquer" in
some sense.
----
Best Wishes,
Vladimir
|
|
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
Vladimir Trushkin
Guest
|
| Posted: Wed May 30, 2007 6:08 pm Post subject: Re: Testing in large scale projects |
|
|
On May 30, 4:37 pm, "David Lightstone"
<david._NoSpamlightst...@prodigy.net> wrote:
| Quote: |
Such assumptions are present, but when the deadlines approach risk
mitigation is activated (often independently developed on the fly). These
envariably entail change to the scope of the deliverable, change to the
scope of the testing that will be performed, etc. The various risk
mitigation plans need not necessiarly be consistent or coordinated. Ergo you
don't know what your will be working against. Although described as
interative (initially) it degenrates to diffusive. That is once thrown over
the wall, the recipeint gets to characterize and adapt
|
I've been working in a project like this. Honestly, we did everything
possible but failed almost every release in a row. It was always a
challenge to get the system working. Test design was very often
performed in parallel with test execution. It was quite common to find
something that invalidates the efforts previously applied, the
workload was drifting toward the deadlines, etc. Overtime, poor
quality and blame from the top management was all we achieved.
My personal belief is that all this starts from overly optimistic
expectations and bad communication of risks and technical limitations
back to the management. Good realistic schedule can only be developed
in a dialog between the stakeholders and shall never be dictated by
one side. Development brings in the technical perspective when
management stands for the business interests. Unfortunately technical
people are not always good at communicating their ideas, working
managers on the matters of estimates. What one needs to keep in mind
that there is no "you" and "them" in such a discussion. Developers and
Managers have the same strategic goals and it's better to think of how
to achieve the goals with available resources and times. From my own
experience, there is always a way.
If management does not accept your estimates then try to do estimates
with more than one method and do a presentation of the way of
estimates and the results. If even this does not work ask for
permission to hire a consultant that will verify your estimates
independently. If you management still believe you are unique and can
do twice more than other bright people in the industry I am afraid
there is no other option but to quit.
----
Best Wishes,
Vladimir |
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
David Lightstone
Guest
|
| Posted: Thu May 31, 2007 4:39 am Post subject: Re: Testing in large scale projects |
|
|
"Vladimir Trushkin" <Vladimir.Trushkin@gmail.com> wrote in message
news:1180534104.017920.110050@q69g2000hsb.googlegroups.com...
| Quote: |
On May 30, 4:37 pm, "David Lightstone"
david._NoSpamlightst...@prodigy.net> wrote:
Such assumptions are present, but when the deadlines approach risk
mitigation is activated (often independently developed on the fly). These
envariably entail change to the scope of the deliverable, change to the
scope of the testing that will be performed, etc. The various risk
mitigation plans need not necessiarly be consistent or coordinated. Ergo
you
don't know what your will be working against. Although described as
interative (initially) it degenrates to diffusive. That is once thrown
over
the wall, the recipeint gets to characterize and adapt
I've been working in a project like this. Honestly, we did everything
possible but failed almost every release in a row. It was always a
challenge to get the system working. Test design was very often
performed in parallel with test execution. It was quite common to find
something that invalidates the efforts previously applied, the
workload was drifting toward the deadlines, etc. Overtime, poor
quality and blame from the top management was all we achieved.
My personal belief is that all this starts from overly optimistic
expectations and bad communication of risks and technical limitations
back to the management. Good realistic schedule can only be developed
in a dialog between the stakeholders and shall never be dictated by
one side. Development brings in the technical perspective when
management stands for the business interests. Unfortunately technical
people are not always good at communicating their ideas, working
managers on the matters of estimates. What one needs to keep in mind
that there is no "you" and "them" in such a discussion. Developers and
Managers have the same strategic goals and it's better to think of how
to achieve the goals with available resources and times. From my own
experience, there is always a way.
If management does not accept your estimates then try to do estimates
with more than one method and do a presentation of the way of
estimates and the results. If even this does not work ask for
permission to hire a consultant that will verify your estimates
independently. If you management still believe you are unique and can
do twice more than other bright people in the industry I am afraid
there is no other option but to quit.
|
I have a much simpler explination. It's basis is articulated in a book
titled Artists, Craftsmen and Technocrats.
There is an impedance mismatch. Doesn't matter what your estimates are. The
only thing that matters is whether you "speak" the language of your
management. If you don't you have no standing, only their ill-formed opinion
stands. If you do, well you can negociate for something that at least might
be reasonable
| Quote: |
----
Best Wishes,
Vladimir
|
|
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
Vladimir Trushkin
Guest
|
| Posted: Thu May 31, 2007 10:04 am Post subject: Re: Testing in large scale projects |
|
|
On May 31, 3:39 am, "David Lightstone"
<david._NoSpamlightst...@prodigy.net> wrote:
| Quote: |
I have a much simpler explination. It's basis is articulated in a book
titled Artists, Craftsmen and Technocrats.
There is an impedance mismatch. Doesn't matter what your estimates are. The
only thing that matters is whether you "speak" the language of your
management. If you don't you have no standing, only their ill-formed opinion
stands. If you do, well you can negociate for something that at least might
be reasonable
|
This is what I mentioned as not being able to communicate the
consequences clearly. Most of managers will not believe you until
convert what you are saying into the terms they can get. All the
managers understand cost, quality and functionality dilemma. All you
need is to honestly present all the facts and ask them what they may
accept to sacrifice.
However if you have no voice then you are hardly in position to change
anything.
----
Best Wishes,
Vladimir |
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
David Lightstone
Guest
|
| Posted: Thu May 31, 2007 4:53 pm Post subject: Re: Testing in large scale projects |
|
|
"Vladimir Trushkin" <Vladimir.Trushkin@gmail.com> wrote in message
news:1180597035.524717.323130@k79g2000hse.googlegroups.com...
| Quote: |
On May 31, 3:39 am, "David Lightstone"
david._NoSpamlightst...@prodigy.net> wrote:
I have a much simpler explination. It's basis is articulated in a book
titled Artists, Craftsmen and Technocrats.
There is an impedance mismatch. Doesn't matter what your estimates are.
The
only thing that matters is whether you "speak" the language of your
management. If you don't you have no standing, only their ill-formed
opinion
stands. If you do, well you can negociate for something that at least
might
be reasonable
This is what I mentioned as not being able to communicate the
consequences clearly. Most of managers will not believe you until
convert what you are saying into the terms they can get. All the
managers understand cost, quality and functionality dilemma. All you
need is to honestly present all the facts and ask them what they may
accept to sacrifice.
However if you have no voice then you are hardly in position to change
anything.
|
We are no addressing a different subject than my initial posting.
Yes and no. You have presented the perspective of the technocrat. This works
quite well in the presense of certainty about the "facts". It fails when
uncertainties are present. The impedance mismatch relates to how they handle
uncertainty.
| Quote: |
----
Best Wishes,
Vladimir
|
|
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
David Lightstone
Guest
|
| Posted: Fri Jun 01, 2007 2:37 pm Post subject: Re: Testing in large scale projects |
|
|
"Vladimir Trushkin" <Vladimir.Trushkin@gmail.com> wrote in message
news:1180685778.868688.26860@k79g2000hse.googlegroups.com...
| Quote: |
On May 31, 6:20 pm, "David Lightstone"
david._NoSpamlightst...@prodigy.net> wrote:
Uncertainty decreases only when measurable accomplishments are achieved
and
coordinatable with other accomplishments. When such coordination is
lacking, glue type efforts are often initiated as stop gap efforts. These
stop gap efforts increase the uncertainty, because they are not
coordinated
(ie characteristics of stop gap my not be appropriate)
No coordination between achievements sounds like a bad planning or
having no plan at all. If there is a plan it's just a formality. I
wonder if one can do something about it. As they say, a fish starts
spoiling from the head (sorry could not find an English analog).
|
The English equivalent is generally state as - a fish starts rotting from
its head.
It do not think the statement that I made about uncertainty and
accomplishments (above) is specific to the particular corporation alone.
That is I believe it to be a characteristic of large projects having both
aggressive schedules and inadequte staff. It contradicts a belief (held by
you) and perhaps others, that uncertainty is usually eliminated as projects
are developed. The particular phenomena here related not to general task
coordination, but rather to miscoordination of various risk mitigation
plans.
Some things just don't scale well. Project planning seems to be one.
| Quote: |
----
Best Wishes,
Vladimir
|
|
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
Vladimir Trushkin
Guest
|
| Posted: Fri Jun 01, 2007 3:01 pm Post subject: Re: Testing in large scale projects |
|
|
On Jun 1, 1:37 pm, "David Lightstone"
<david._NoSpamlightst...@prodigy.net> wrote:
| Quote: |
It do not think the statement that I made about uncertainty and
accomplishments (above) is specific to the particular corporation alone.
That is I believe it to be a characteristic of large projects having both
aggressive schedules and inadequte staff. It contradicts a belief (held by
you) and perhaps others, that uncertainty is usually eliminated as projects
are developed. The particular phenomena here related not to general task
coordination, but rather to miscoordination of various risk mitigation
plans.
Some things just don't scale well. Project planning seems to be one.
|
Yes, many things do not scale well. When one gets back from a
supermarket with lots of buying does he try to bring it all home
altogether? Usually not, he will try to pick up a handful of load at a
time. The same is true for the large projects. They must be divided
into smaller parts in order to be more controllable. In our
organization we noted that the percentage of failed tests in system
testing is directly affected by now many requirements are delivered.
The more requirements we get the more often tests fail.
If you say that your [project is understaffed then you have already
made estimates and they appear to be bigger than planned. This fact
puts you on the risk of missing the deadline. In order to communicate
this fact to the management you need something more than just a
statement "we are not going to make it"... All you need is to
communicate your vision clearly and draw the consequences if your
vision will be ignored.
----
Best Wishes,
Vladimir |
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
David Lightstone
Guest
|
| Posted: Fri Jun 01, 2007 7:25 pm Post subject: Re: Testing in large scale projects |
|
|
"Vladimir Trushkin" <Vladimir.Trushkin@gmail.com> wrote in message
news:1180695670.443380.84470@o5g2000hsb.googlegroups.com...
| Quote: |
On Jun 1, 1:37 pm, "David Lightstone"
david._NoSpamlightst...@prodigy.net> wrote:
It do not think the statement that I made about uncertainty and
accomplishments (above) is specific to the particular corporation alone.
That is I believe it to be a characteristic of large projects having both
aggressive schedules and inadequte staff. It contradicts a belief (held
by
you) and perhaps others, that uncertainty is usually eliminated as
projects
are developed. The particular phenomena here related not to general task
coordination, but rather to miscoordination of various risk mitigation
plans.
Some things just don't scale well. Project planning seems to be one.
Yes, many things do not scale well. When one gets back from a
supermarket with lots of buying does he try to bring it all home
altogether? Usually not, he will try to pick up a handful of load at a
time. The same is true for the large projects. They must be divided
into smaller parts in order to be more controllable. In our
organization we noted that the percentage of failed tests in system
testing is directly affected by now many requirements are delivered.
The more requirements we get the more often tests fail.
If you say that your [project is understaffed then you have already
made estimates and they appear to be bigger than planned. This fact
puts you on the risk of missing the deadline. In order to communicate
this fact to the management you need something more than just a
statement "we are not going to make it"... All you need is to
communicate your vision clearly and draw the consequences if your
vision will be ignored.
|
The word estimate as you use it implies quantitative. As your management
uses it, it also implies quantative. When someone tells me "we are not going
to make it", that is adequate. That is a definitive statement made by a
professional. The only question is whether the word of a professional is
meaningful or not. Were there doubt they would qualify as appropriate.
The only reason for further investigation as to why "we are not going to
make it" relates to resource allocation. This investigation is only
meaningful when there was a deliberate effort to understaff or under
resource a project. The intent of the investigation would be to see if a
reconfiguration of staff or resource might make things possible. That is the
original plan was expected to fail (but hoped to be successful in a wishful
thinking sense), failure was acceptable and recovery was deemed to be
possible. The investigation serving to establish the recovery strategy.
I do not particularly like your "communicate clearly" argument. It
completely misses the managerial gamesmanship issue (indicated above)
| Quote: |
----
Best Wishes,
Vladimir
|
|
|
| Back to top |
|
|
| |
Ads |
Advertising
Sponsor
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
212 Attacks blocked
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
