Visibility into the FPGA.

Posts Tagged mehodology

Get your money back in 4 weeks

Exostiv pays back in 2 weeks

Get your money back in 4 weeks

Debug productivity is notoriously hard to sell.

Engineers who ask budgets for debugging tools are still too often blamed by the Management for creating the bugs in first instance.
(‘Why would I pay more for correcting the bugs that YOU inserted in the design?’).

Putting a value on debugging is a particularly hard task… It is all about reducing the time spent on debugging but how much does it really cost and how can we be sure that a specific tool really brings an improvement?

Saving how much of the engineering time would justify buying EXOSTIV?
A rule of thumb is 4 weeks per year (or even as low as 2 weeks for the lucky ones located in high salary areas).

A good example

I recently visited a company where the engineering team wanted to evaluate EXOSTIV on an existing board. This board was mounted with an FPGA supported by EXOSTIV and featured a single SFP connector. As such, it was usable ‘out of the box’. We offered to set up the project files for EXOSTIV with the engineering and within 30 minutes, we could insert a debug EXOSTIV IP into the target design. As we did it ourselves, there was no initial setup cost nor learning curve cost in this example. After FPGA implementation, EXOSTIV connected right away and we could capture data. A good demonstration as it seemed. After 2 hours, I left the engineering team with a trial unit of EXOSTIV and allowed them to use it for free until the next day.

The next day, the engineering team told me that the tool was easy to use for those used to JTAG-based logic analyzers such as Chipscope / Xilinx logic analyzer. Basically, the flow was identical. Configuring transceivers required some additional experience to understand the metrics, clock sources and so on, but this was general knowledge of FPGA structure that any engineer should learn someday.

Then, they told me that the visibility provided by Exostiv had allowed them to find and correct a bug in an Ethernet IP, that they had not been able to see before, because their tools could not reach such debug scenarios. They were about to go to production and said that the result was ‘invaluable’. This result had totally exceeded my expectations.

I was absolutely delighted.
I expected to receive a purchase order the same day.

I was wrong.

When ‘invaluable’ kills business

Actually, they were puzzled. They somehow went to the conclusion that EXOSTIV was priced too high because our model involves subscribing for EXOSTIV software for a minimum of 12 month – and here the bug resolution had been so fast… (I am still perplexed by this reasoning…). Anyway, they decided to wait until they had a new bug or alert that could justify buying the tool
EXOSTIV had revealed an issue that they were not aware of – and before being painful to anyone.

And what about the management? Practically, nothing harmful had happened at all – so the management was not even considering a purchase…

Missed market opportunity cost

Going to production with unknown bugs has a cost that generally reduces to how much market (share) you’ll loose by arriving late on the market with a working product. In this case, it seemed that the product was already reasonably stable: the engineering team was perfectly qualified and had not seen anything wrong.

This cost is called ‘(missed) market opportunity cost’ and can be estimated at the value of the market that is left to the competitor because you are delaying your product launch. Even if this cost can be large (loosing a few % of market share should be a lot of money – or you do not address a market that is large enough), it can have no impact on a decision to invest in a new EDA tool to debug FPGA. The value can hardly be estimated accurately and its consequences are usually unpredictable and too distant. Much too complicated…

Bottom line: ask the right questions

– Will there be bugs in your design? Absolutely. FPGA are such complex beasts that this cannot be avoided. No wonder why 40% of the total design time is spent on debug and verification.

– When do those bugs cost the most? When they ‘escape’ to production: the cost of having to stop the production and get back to design is gigantic. And it si your responsibility as an engineer to find them.

– Can EXOSTIV help you find them? You bet. EXOSTIV provides unprecedented visibility.

And finally:

– Why would you reserve a budget for EXOSTIV? Because it pays back if you save 4 weeks of engineering per year. And this can be 4 weeks total for a team that shares one license.

Thank you for reading.
– Frederic

The FPGA problem we are trying to solve

Problem and solution

The FPGA problem we are trying to solve

Designing FPGA can be complex. Each step of the design flow brings its own challenges, problems and solutions. As engineers, this is what we do: we find solutions. We use our knowledge, we mobilize our skills and find the right tools to constantly build better solutions to the problems that we encounter.

Exostiv Labs aims at providing better tools for FPGA debug. However, ‘FPGA debug’ may be understood quite diversely. In this post, I’d like to go back to the basics and pinpoint the specific problems we are trying to solve.

‘First time right’ at board bring-up is a problem.

I am an engineer. Like any engineer, I like when things are done right technically, with the right steps taken at the right time.
However, as an engineer, I also have learned to manage budget and market constraints. We all know that between the ‘perfect design flow’ and the ‘actual design flow’, the distance is made of budget and/or market constraints. We all know that sometimes, it is chosen to skip the verification of one ‘detail’ in order to reach the announced release date. Sometimes it is expected from us to accept a small percentage of uncertainty (that is called ‘experience’ or ‘know-how’) because it is statistically cost-effective.

Ideal vs. Actual design flow

“And what if a bug appears later? Well, we’ll fix it and send an upgrade.
Isn’t it the beauty of programmable devices?”

Sometimes, being ‘first on the market’ or ‘first in budget’ beats ‘first time right’.

A typical in-lab situation

We all have been there…

The system starts up and suddenly, ‘something goes wrong’. It happens randomly, sometimes 2 minutes, sometimes 2 hours after power up.
– We have no clue about what caused the bug.
– We have no idea about why it happens.
– We do not know the time from cause to effect.

This is called ‘an emergent system with a ‘random bug’.

It is emergent because it is the result of complex interactions of individual pieces. Such system-level bugs are the most complex – and time-consuming – to solve because they involve the complex interactions of a whole system.

EXOSTIV solves the problem of finding the roots of bugs appearing in complex systems placed in their real environment.

EXOSTIV solves FPGA board bring up problems

Solving such a problem requires *a LOT of* visibility on the FPGA under test:
– It is hard to put a trigger condition on an unknown random bug. To spot it, you’d better have the largest capture window possible.
– You sometimes need to extend the capture far in time (stressing the impracticability of simulation as sole methodology).
– What happened before seeing the bug is very important to understand the ‘fatal’ sequence of events that has created the faulty condition.

Without a sufficient observability of the system under test, you’ll simply loose precious engineering hours hoping to -just- capture the bug. Then you’ll spend time again trying to capture the key events that happened before it – that ultimately would help you understand why the bug occurs.

Instrumenting a real design is the key to overcoming modelling mistake and see bugs occurring in the real environment… but only if the instrumented design provides sufficient visibility !

What can you do with 8 GB captured at FPGA speed of operation?

Please think to it. I’ll explore how to make the most of the *huge* resources offered by EXOSTIV in my next post.

Thank you for reading
– Frederic

‘My FPGA debug and verification flow should be improved…’

Improve the FPGA debug flow

‘My FPGA debug and verification flow should be improved…’

In my last post, I revealed some of the results of our recent survey on FPGA. These results depicted a ‘flow-conscious’ FPGA engineer, using a reduced mix of methodologies in the flow and very prone to going to the lab for debugging.

In the same survey, we tried to evaluate the level of satisfaction of the FPGA engineer for his/her debug and verification flow. We asked the respondents to select among several propositions the one that the most closely matched their thinking. See the picture below (click on picture to zoom).

Recognition of the need to improve the FPGA debug flow

More than 70% of the respondents recognize the need to improve the FPGA debug and verification flow.

The chart above represents the answers of the respondents active with an FPGA design, and actually using FPGA as a target technology. 72% of them recognize the need to improve the FPGA debug flow and nearly 40% of them are actively looking for a solution for it.

Are these survey results representative of the whole industry? Well, you tell me. Contact me to share your personal experience – and I’ll update this post.

– Oh, and by the way, as I write this, we are about to release our EXOSTIV solution. It improves the visibility on FPGA running at speed by a factor of up to 200.000 ! See below a preview of what we’re about to release. More information will be available soon.

EXOSTIV Dashboard software screenshot preview

Thank you for reading.
– Frederic

Does FPGA use define verification and debug?

Does FPGA use define verification
and debug?

You may be aware that we have run a first survey on FPGA design, debug and verification during the last month.

(By the way, many thanks to our respondents – we’ll announce the Amazon Gift Card winner in September).

In this survey, we did some ‘population segmentation’, and first asked about the primary reason for using FPGA:

– Do you use FPGA as a target technology? In other words are FPGA part of your final product’s BOM (bill of materials) list? … – or:
– Do you use FPGA as a prototyping technology for an ASIC or a SoC… ?

49 respondents were found in the first group (which we’ll also call the ‘pure FPGA’ users) and 34 in the second group. The survey was posted during about one month online and advertised on social networks. The questions were in English.

Behind this segmentation question is of course the idea that the design flow used to design FPGA is usually significantly different (and shorter) than this used to design an ASIC.
(This is claimed by FPGA vendors as one of the significant differences between ASIC and FPGA – you can see an example of this comparison here).

Since we care (a lot) about FPGA debug and verification, one of the survey questions was about the set of tools currently used for FPGA debug and verification

The chart below illustrates the percentage of respondents in each group who selected each proposed tool or methodology. They had been invited to select all choices that applied to them (click on the picture to enlarge).

Survey results

A podium shared by simulation, in-lab testing and RTL code reviews

Without significant difference between the 2 groups, simulation is the clear winner methodology, with a little more than 90% use. Frankly, we wonder how it is still possible not to simulate a FPGA design today – we have not analyzed the cases when designers are able to skip this step.

In-lab testing comes second, with near to 70% usage among the respondents. In-lab testing actually includes 2 categories of tools in the survey:

– traditional instruments -such as scopes and logic analyzers- and
– embedded logic analyzers -such as Xilinx Chipscope, Altera Signal Tap and the likes.

Manual RTL code review closes the podium. We expect that any designer actually reviews his/her code. Why is it not 100%? Probably because the ones verifying and debugging FPGA are not the RTL coders. Another possibility is that ‘reviewing the code’ might not be seen by many as a ‘verification technique’, but as the result of the process of verification and debug.

The case of code coverage

While this technique has been around for quite some time now (and is actually complementary to simulation), it is interesting to see that code coverage has got the same popularity in our 2 groups (we do not think that the difference between the groups is significant here).

It has been widely said that ‘pure FPGA designers’ are careless and do not use the same advanced and rigorous techniques of ASIC designers. Well, apparently, they do not just care about simulating some cases; they have the same level of concern for properly covering the code as ASIC designers.

Assertions: YES
UVM : not for ‘pure’ FPGA designers

UVM is quite a hot topic today, with most of the major EDA companies actively promoting this methodology. For those unfamiliar with the subject, here is brief overview. It seems to currently have a quite positive upwards trend for ASIC and SoC design. Tools and environments are appearing on the market, that can be applied to FPGA design too. Like we see on the chart, UVM has not really gained traction among our ‘pure FPGA’ respondents, although near to one engineer on 5 among our ‘ASIC engineers’ has adopted this technique.

Takeaways

1) In-lab testing is a well-entrenched habit of the FPGA designer. And there seems to be no notable difference between our groups. Like we pointed out in a previous publication, in-lab FPGA test and debug is here to stay. We simply believe that – for FPGA at least – this technique must and can be improved.

2) The FPGA engineer seems to be using a mix of methodologies when it comes to debug and verification, but not necessarily a lot of them.
Among our respondents, we have found that the average set of techniques used for debug and verification counts 2.58 different techniques (2.39 for the ‘pure FPGA players’ against 2.68 for the ASIC designers).
And – please note: simulation is there of course, often completed with code coverage.
‘The FPGA engineer who does not simulate at all’ appears to be just a myth today…

3) There is no notable difference between designers using FPGA as a target technology and those using FPGA as a prototyping technology- except for:

– Equivalence checking -and:
– UVM
… which seem to have a very limited adoption among the ‘pure FPGA players’.

In a later post, we’ll further analyze the results of our survey. We have shown which techniques are currently used for debug and verification.

Of course, the million dollar question is whether FPGA engineers are satisfied with this situation or if they are in need of improvements… Stay tuned.

Thank you for reading.
– Frederic

STAY IN TOUCH

Sign in to our Newsletter