SparkFun Forums

[Valen]

You run test procedures to find bugs. You don't run testing procedures just for the fun of it. Only in rare chances would you find a bug by random pushing buttons or whatever. So yeah, sometimes it is beneficial to not follow the testing procedures and just be stubborn and do anything to the device under test.

If every thing is working as it should then you will not find bugs. Or the testing procedures were not deep enough, and the bug is more obscure. You don't catch something with fishing every time either. I guess the creativity comes in finding the loopholes in those tests. Figuring out what this test is not actually testing.

Let me answer your questions with a return question. Did you ever miss a bug that others did find? Did they use different testing methods. And if so, how did their method compare to your way of doing things?

[omega supreme]

You run test procedures to find bugs. You don't run testing procedures just for the fun of it. Only in rare chances would you find a bug by random pushing buttons or whatever. So yeah, sometimes it is beneficial to not follow the testing procedures and just be stubborn and do anything to the device under test.

If every thing is working as it should then you will not find bugs. Or the testing procedures were not deep enough, and the bug is more obscure. You don't catch something with fishing every time either. I guess the creativity comes in finding the loopholes in those tests. Figuring out what this test is not actually testing.

Let me answer your questions with a return question. Did you ever miss a bug that others did find? Did they use different testing methods. And if so, how did their method compare to your way of doing things?

Most bugs are found by end user testing, which I suck at. I feel like I run test procedures like a monkey. I try to run a lot of tests but I almost never find bugs.

[Valen]

End-users finding more bugs than the testing/QA department doesn't surprise me. The end-users are in large(r) numbers. So have more testing time as a whole and will try to do things in almost infinitely different ways.

Also, I think production variances in materials and assembly of devices can help them uncover more bugs if it pertains to the interaction of hardware and software. You know, i.e. capacitors and resistors making a bit larger timeconstants than the software expects as a time out. You will not likely experience that with just a handfull of devices in your test lab.

What I dislike about this end-user testing thing is that companies might think of this as a way of crowd-sourced QA testing. Which would allow then to cut their budgets on product testing and get to the market a bit quicker. I really do not want to do the work of the testing team, after I bought something and doesn't seem to work right. But that is a soapbox, which I should step down from.

[omega supreme]

I'm afraid the end users are going to make me obsolete as a procedure tester.

[omega supreme]

Maybe hardware testing is easier because they don't have to worry about bug counts. They just run procedures. But some of those procedures are thick. Hardware testers seem to work weekend and overtime. I want to party and have fun life.

[jcorpey]

I come from a graphic design background and recently started taking up electronics as a hobby. When I first started in the design field, while in school, felt like everyone around me was more skilled and more creative than I was. I started really cracking down thinking that perhaps being more technically skilled might help to compensate where my creativity was lacking. But now that I am working in a successful design career, I realize that improving your technical skills in parallel can improve your creativity.

Everyone can learn to be creative, just like everything else, it takes practice.

[omega supreme]

Maybe if I keep studying the specs and test procedures I will eventually have a breakthrough of understanding?

[lyndon]

Which is more important? Finding bugs or running test procedures? I've tried end user and ad hoc testing and I can't seem to find any bugs. Can I be an automaton just running test procedures? Will I get fired or laid off by just being strictly a procedure tester?

Either way you still have to run procedures. If you find a bug, how will someone replicate it if you don't tell them exactly what you did? (one of my pet peeves about some testers I've worked with) How will anyone know after 5 years of changes if the bug has been reintroduced if there aren't regression procedures to run?

neildarlow is correct in that there are two main tasks of testing and two very different personality types that excel at each.

[omega supreme]

My supervisors says to write up everything. I write almost all 'Once' issues. I just take logs. I feel bad I don't know what triggered the problem.

[lyndon]

You are really not being very clear about what you do!

What does "I just take logs" mean? If you are a tester, then what did you do to find a defect? Are you following a process or just randomly poking around? Finding bugs isn't very useful unless you can tell someone how to recreate the bug.

[omega supreme]

You are really not being very clear about what you do!

What does "I just take logs" mean? If you are a tester, then what did you do to find a defect? Are you following a process or just randomly poking around? Finding bugs isn't very useful unless you can tell someone how to recreate the bug.

I test software. I run procedures that others create. If some bug occurs randomly I can capture logs built into the code and include it in my problem report. If I didn't write once issues (ie. with no steps to reproduce) I would have a zero bug count. Bug count is the main performance metric. I'm known for pushing test progress but I run the same procedures over and over again, the ones I feel comfortable with and can get done fast. I stay away from the new feature, complex tests. I have a bad reputation for cherry picking the short, easy tests to inflate my test count. How can I grow to become more than just a monkey tester, automaton?

[ImpGuru]

Honestly, it sounds like you are working for a company who doesn't respect you, if what you say is true, this seems like a high stress environment which in and of itself would prevent creativity.

In any case to become creative (it can definitely be done) you need to dig into WHY the procedures are what they are. You need to develop a working model in your mind for cause and effects of the thing that you are testing. Why does it work like that, does it make sense? Think about the customers, put yourself in their shoes what they would do and think. Ask your boss for career development opportunities, if they value you as a employee, they will invest in making you more effective.

[omega supreme]

Which is more important for me to read/study, the specs or test procedures? I want to know the best place to invest my time and effort.

[Ross Robotics]

I am sorry for being the one that's negative, but if you have to ask what to read/study, then you will always be the "Monkey." Why not just read/study everything you can?

[omega supreme]

I can't remember anything so it's not possible for me to learn everything. I need to be selective. I apologize if it sounds like I'm making excuses, which are tools of the incompetent.