Exostiv boosts RTL simulation

By Frederic Leens on September 5, 2018 / Uncategorized

Tags: analysis, ASIC, ASIC prototyping, Design Flow, emulation, flow, FPGA, model, modelling, prototyping, RTL simulation, simulation, snapshot, SoC, specification, tools, UVM, UVVM, Verification, visibility

Exostiv boosts RTL simulation

It is essential to reduce the wasted machine cycles used for simulation workloads.

Simulation dominates ASIC/SoC/FPGA verification process

‘The 2018 Wilson Research Group ASIC and FPGA Functional Verification Study’ (you may have to register for free to watch this) reports that an ASIC, SoC or FPGA designer can spend up to 40% of the total verification time on creating testbenches, writing tests or running simulation. In addition up to 44% of the time spent on debug – and this involves running simulation too.
It is safe to say that simulation is the most dominant technique in the overall verification effort.

Simulation speed is losing the race against FPGA complexity.

Over the last decade, simulation software has improved at many levels. Simulation software needed these improvements so they would be able to cope with the increasing complexity of designing FPGAs and chips with hundreds of millions gates.

Simulation software now supports more languages, assertions, and verification methodologies such as Accellera UVM. Code coverage now includes testbench coverage – in order to keep track of whether a team has exercised specific test stimuli.

Interestingly, these efforts have improved the test coverage, provided metrics for signoff and standardized verification approaches based on simulation, but did not accelerate simulation that much. Even the most recent claims from simulation software vendors show RTL simulation acceleration by 4x or maybe 5x thanks to the execution of the simulation on multi-core servers… whereas the complexity of FPGA has been multiplied by 100 in the same period of time!

In 2018, 84% of FPGA designs had at least 1 non-trivial bug escape to production (source: The Wilson Research Group)

How to make the most of the simulation time

In the study mentioned above, The Wilson Research Group reports that a staggering 84% (!) of the FPGA designs have had at least 1 non-trivial bug escape to production – and that: 65% of the typical simulation times exceed 5 hours.

That is why it has become critical to make the most of the time spent in simulation, and limit – as much as it is possible – the wasted machine cycles. Otherwise, our verification techniques are at risk of becoming irrelevant.

In our experience, the following approaches have proven to be successful:
– Performing ‘per IP’ simulations to verify them helps spread the effort on multiple engineers in parallel and run shorter simulations;
– Make use of ‘snapshot’ simulation features – which consists in saving the state of a simulation and re-start from there to run specific simulations.

In the illustration below, two frequent snapshots are saved – after the boot of the processor and after the DUT is initialized respectively. This conveniently helps running combinations of initialization sequences and test stimuli without having to simulate the boot sequence over and over again.

Simulation shnapshot technique

Full system bugs can require multiple days of simulation.

As demonstrated in this presentation from Cadence on simulation tools, full-chip bugs, which occur when the full system is put together, can require days of simulation. A ballpark estimate is up to 54 hours of simulation for 43 milliseconds of SoC HW runtime! For these kinds of bugs, it has become equally crucial to:
– Speed up simulation; (that’s the job of simulation software vendors)
– Make sure that the simulated models are accurate; (EXOSTIV can definitely help !)
– Provide additional visibility from the real system to converge faster on finding the bug; (EXOSTIV can definitely help!)

What EXOSTIV can do to BOOST simulation.

Exostiv lets you record very long data sets from inside FPGAs running at full speed of operation.
Typical probing scenarios with their most common usages are summarized in the two tables below:
The first table lists common usages that boost simulation – illustrated below the table:

What is probed	Target usage
Specific IP inputs (or group of IPs)	- Overcome modelling issues; - Make test case match reality.
Specific IP outputs (or group of IPs)	- Reference response database; - See the effects of non- or hardly simulable elements, (such as CDC or metastability).

Exostiv lets you record input vectors from reality for IPs and full systems

Other Exostiv usages that complement simulation:

What is probed	Target usage
A full system bus	- Verify / develop software; - Evaluate system overall efficiency / performance over large - and real - data; - Check system behaviour;
Specific nodes	- Functional verification and debug.

Conclusion

Exostiv complements simulation by providing a source of real-world stimulus for IPs, groups of IPs and full systems. It therefore helps overcome modelling problems that are the most common source of FPGA bugs escaping to production. Coupling simulation with the huge recording capability of Exostiv means you avoid wasting your quota of machine cycles dedicated to simulation and improves the quality of FPGA system sign-off.

When 84% of the FPGA designs report at least 1 non-trivial bug escaping to production, and 65% of the typical simulation times exceed 5 hours (source: The 2018 Wilson Research Group ASIC and FPGA functional verification study), boosting simulation is an absolute necessity.

As always, thank you for reading.
– Frederic

You can capture tons of data. Now what?

By on January 22, 2016 / Uncategorized

Tags: brute force, burst, capture size, capture window, data qualification, FPGA, in-lab debug, observability, random bug, tools, transceiver, trigger, visibility

EXOSTIV can capture very large data sets

You can capture tons of data. Now what?

Offering huge new capabilities is not always seen positively. Sometimes, engineers come to us and ask:
‘Now that I am able to collect Gigabytes of trace data from FPGA running at speed… how do I analyze that much data?’.

Does EXOSTIV just displace the problem?

No, it does not.

In this post, I’ll show:
1) why it is important to enlarge the observation window. – and:
2) how to make the most of this unprecedented capability, based on EXOSTIV Dashboard’s capture modes, triggers and data qualification capabilities.

Random bugs happen

The figure below shows a typical debug scenario. A bug occurs randomly ‘some time’ after system power up. During an initial time interval, the effects of the bug are not noticeable – after which, the system’s behaviour is sufficiently degraded to obviously see the bug’s effect. Typically, the engineer will need to detect a ‘start of investigation’ condition from which he’ll have to ‘roll back in time’ – ultimately to find the root cause of the bug. In the case of an FPGA the engineer observes the FPGA’s internal logic.

Random bugs typically occur as a result of a simulation model issue. The system’s environment could not be simulated with complete accuracy and/or some part of the system behaves differently than initially thought int he ‘real world’. In addition, long operating times are not really the ‘simulation’s best friend’.

Size matters.

Suppose that we capture some ‘observation window’ of the system. If we are lucky enough to observe a bug right when it occurs, we can get an understanding of what happens. Similarly, we are able to recognize the effects of the bug, as shown by the orange ‘X’ on the picture (called ‘start of investigation). Conversely, we have no idea about how to trigger on such conditions (if we had the ability to trigger on unknown and random bugs, they would have been corrected already).

With a reduced observation window, the process of debugging requires some luck. We can miss everything, recognize an effect without having sufficient data to find the start of investigation. We can be lucky enough to find the start of investigation but no history to roll back to the root cause… – Or: we can hope to ‘shoot right on the bug’.

You do not want to leave the debugging process to chance?

Enlarge the observation window!

VS.

With a properly sized observation window, you can be certain of finding a random bug.

Not ‘brute force’ only

EXOSTIV provides a total observation window of 8 Gigabytes.
However, EXOSTIV is not a ‘brute force’ solution only. Many of its features are equally important to implement ‘smarter’ captures in the lab.

Trigger, Data Qualification, Streaming and Burst help you define better capture windows.

1. Trigger

The total capture window may be gigantic, a triggering capability remains essential. During EXOSTIV IP setup, the nodes that need to be observed are optionally marked as ‘source for trigger’. Currently, EXOSTIV is able to implement a single or repeating trigger based on AND and OR equations defined on these signals. The AND and OR equations can also be combined. (In a future release, EXOSTIV will provide sequential triggers as well.)

With the growing complexity of systems, defining a very accurate trigger condition has sometimes become a way to overcome a too limited capture window.

This time is over. You are now able to use trigger to collect a better capture window. Maybe you can relax a little bit about defining complex triggering, because you can count on a very large capture window to collect the information that you need.

2. Data Qualification

‘Data qualification’ enables you to define a logic condition on the observed signals. Data is captured and recorded by EXOSTIV only when this condition is true.

For instance, this can be used to filter out the times when nothing happens on a bus. This is a very powerful feature that helps you preserve the bandwidth that you use to extract data.

3. Streaming or Burst?

EXOSTIV provides 2 modes of capture:

Streaming mode: data is captured continuously. If the bandwidth required to capture data exceeds its EXOSTIV Probe capabilities, it stops and resumes capture once it is able to do so. Triggers and capture length can be used to define a repeating start condition.
Burst mode: data is captured by ‘chunks’ equal to the ‘burst size’. The maximum burst size is defined by the size of the Capture Unit’s FIFO (click here for more details about EXOSTIV IP). So, ‘burst mode’ is always an ‘interrupted’ type of capture.

Takeaways

Using EXOSTIV reduces to 2 types of capture: continuous or interrupted.
‘Continuous capture’ is easy to understand: EXOSTIV starts capturing data and has sufficient bandwidth on its transceivers to sustain data capture until its large probe memory is full.

This is an unprecedented capability for a FPGA debug tool.

‘Interrupted capture’ – whether based on trigger, data qualification, burst mode or a combination of them – help ‘better capture windows’ that focus on data of interest. Interrupted captures also help go beyond the maximum bandwidth of the transceivers and capture data even farther in time than what a continuous capture would allow.

This considerably extends your reach in time, after starting up the FPGA.

Thank you for reading.
– Frederic

‘My FPGA debug and verification flow should be improved…’

By Didier Martiny on September 25, 2015 / Uncategorized

Tags: Debug, engineering, EXOSTIV, flow, FPGA, in-lab, mehodology, observability, survey, tools, visibility

‘My FPGA debug and verification flow should be improved…’

In my last post, I revealed some of the results of our recent survey on FPGA. These results depicted a ‘flow-conscious’ FPGA engineer, using a reduced mix of methodologies in the flow and very prone to going to the lab for debugging.

In the same survey, we tried to evaluate the level of satisfaction of the FPGA engineer for his/her debug and verification flow. We asked the respondents to select among several propositions the one that the most closely matched their thinking. See the picture below (click on picture to zoom).

More than 70% of the respondents recognize the need to improve the FPGA debug and verification flow.

The chart above represents the answers of the respondents active with an FPGA design, and actually using FPGA as a target technology. 72% of them recognize the need to improve the FPGA debug flow and nearly 40% of them are actively looking for a solution for it.

Are these survey results representative of the whole industry? Well, you tell me. Contact me to share your personal experience – and I’ll update this post.

– Oh, and by the way, as I write this, we are about to release our EXOSTIV solution. It improves the visibility on FPGA running at speed by a factor of up to 200.000 ! See below a preview of what we’re about to release. More information will be available soon.

Thank you for reading.
– Frederic

Does FPGA use define verification and debug?

By on August 20, 2015 / Analysis

Tags: ASIC, assertion, code coverage, Debug, embedded logic analyzer, emulation, equivalence checking, flow, FPGA, in-lab, mehodology, simulation, SoC, survey, tools, UVM, Verification

Does FPGA use define verification
and debug?

You may be aware that we have run a first survey on FPGA design, debug and verification during the last month.

(By the way, many thanks to our respondents – we’ll announce the Amazon Gift Card winner in September).

In this survey, we did some ‘population segmentation’, and first asked about the primary reason for using FPGA:

– Do you use FPGA as a target technology? In other words are FPGA part of your final product’s BOM (bill of materials) list? … – or:
– Do you use FPGA as a prototyping technology for an ASIC or a SoC… ?

49 respondents were found in the first group (which we’ll also call the ‘pure FPGA’ users) and 34 in the second group. The survey was posted during about one month online and advertised on social networks. The questions were in English.

Behind this segmentation question is of course the idea that the design flow used to design FPGA is usually significantly different (and shorter) than this used to design an ASIC.
(This is claimed by FPGA vendors as one of the significant differences between ASIC and FPGA – you can see an example of this comparison here).

Since we care (a lot) about FPGA debug and verification, one of the survey questions was about the set of tools currently used for FPGA debug and verification

The chart below illustrates the percentage of respondents in each group who selected each proposed tool or methodology. They had been invited to select all choices that applied to them (click on the picture to enlarge).

A podium shared by simulation, in-lab testing and RTL code reviews

Without significant difference between the 2 groups, simulation is the clear winner methodology, with a little more than 90% use. Frankly, we wonder how it is still possible not to simulate a FPGA design today – we have not analyzed the cases when designers are able to skip this step.

In-lab testing comes second, with near to 70% usage among the respondents. In-lab testing actually includes 2 categories of tools in the survey:

– traditional instruments -such as scopes and logic analyzers- and
– embedded logic analyzers -such as Xilinx Chipscope, Altera Signal Tap and the likes.

Manual RTL code review closes the podium. We expect that any designer actually reviews his/her code. Why is it not 100%? Probably because the ones verifying and debugging FPGA are not the RTL coders. Another possibility is that ‘reviewing the code’ might not be seen by many as a ‘verification technique’, but as the result of the process of verification and debug.

The case of code coverage

While this technique has been around for quite some time now (and is actually complementary to simulation), it is interesting to see that code coverage has got the same popularity in our 2 groups (we do not think that the difference between the groups is significant here).

It has been widely said that ‘pure FPGA designers’ are careless and do not use the same advanced and rigorous techniques of ASIC designers. Well, apparently, they do not just care about simulating some cases; they have the same level of concern for properly covering the code as ASIC designers.

Assertions: YES
UVM : not for ‘pure’ FPGA designers

UVM is quite a hot topic today, with most of the major EDA companies actively promoting this methodology. For those unfamiliar with the subject, here is brief overview. It seems to currently have a quite positive upwards trend for ASIC and SoC design. Tools and environments are appearing on the market, that can be applied to FPGA design too. Like we see on the chart, UVM has not really gained traction among our ‘pure FPGA’ respondents, although near to one engineer on 5 among our ‘ASIC engineers’ has adopted this technique.

Takeaways

1) In-lab testing is a well-entrenched habit of the FPGA designer. And there seems to be no notable difference between our groups. Like we pointed out in a previous publication, in-lab FPGA test and debug is here to stay. We simply believe that – for FPGA at least – this technique must and can be improved.

2) The FPGA engineer seems to be using a mix of methodologies when it comes to debug and verification, but not necessarily a lot of them.
Among our respondents, we have found that the average set of techniques used for debug and verification counts 2.58 different techniques (2.39 for the ‘pure FPGA players’ against 2.68 for the ASIC designers).
And – please note: simulation is there of course, often completed with code coverage.
‘The FPGA engineer who does not simulate at all’ appears to be just a myth today…

3) There is no notable difference between designers using FPGA as a target technology and those using FPGA as a prototyping technology- except for:

– Equivalence checking -and:
– UVM
… which seem to have a very limited adoption among the ‘pure FPGA players’.

In a later post, we’ll further analyze the results of our survey. We have shown which techniques are currently used for debug and verification.

Of course, the million dollar question is whether FPGA engineers are satisfied with this situation or if they are in need of improvements… Stay tuned.

Thank you for reading.
– Frederic

Posts Tagged tools

Exostiv boosts RTL simulation