Is using double faster than float?
Is using double faster than float?
Double values store higher precision and are double the size of a float, but are Intel CPUs optimized for floats?
That is, are double operations just as fast or faster than float operations for +, -, *, and /?
Does the answer change for 64-bit architectures?
FYI a duplicate question here has some good information also.
– Trevor Boyd Smith
Apr 12 at 17:18
7 Answers
7
There isn't a single "intel CPU", especially in terms of what operations are optimized with respect to others!, but most of them, at CPU level (specifically within the FPU), are such that the answer to your question:
are double operations just as fast or
faster than float operations for +, -,
*, and /?
is "yes" -- within the CPU, except for division and sqrt which are somewhat slower for double than for float. (Assuming your compiler uses SSE2 for scalar FP math, like all x86-64 compilers do, and some 32-bit compilers depending on options. Legacy x87 doesn't have different widths in registers, only in memory (it converts on load/store), so historically even sqrt and division were just as slow for double).
double
float
double
For example, Haswell has a divsd throughput of one per 8 to 14 cycles (data-dependent), but a divss (scalar single) throughput of one per 7 cycles. x87 fdiv is 8 to 18 cycle throughput. (Numbers from https://agner.org/optimize/. Latency correlates with throughput for division, but is higher than the throughput numbers.)
divsd
divss
fdiv
The float versions of many library functions like logf(float) and sinf(float) will also be faster than log(double) and sin(double), because they have many fewer bits of precision to get right. They can use polynomial approximations with fewer terms to get full precision for float vs. double
float
logf(float)
sinf(float)
log(double)
sin(double)
float
double
However, taking up twice the memory for each number clearly implies heavier load on the cache(s) and more memory bandwidth to fill and spill those cache lines from/to RAM; the time you care about performance of a floating-point operation is when you're doing a lot of such operations, so the memory and cache considerations are crucial.
@Richard's answer points out that there are also other ways to perform FP operations (the SSE / SSE2 instructions; good old MMX was integers-only), especially suitable for simple ops on lot of data ("SIMD", single instruction / multiple data) where each vector register can pack 4 single-precision floats or only 2 double-precision ones, so this effect will be even more marked.
In the end, you do have to benchmark, but my prediction is that for reasonable (i.e., large;-) benchmarks, you'll find advantage to sticking with single precision (assuming of course that you don't need the extra bits of precision!-).
This would also depend on the cache block size, correct? If your cache retrieves 64bit or larger blocks, then a double would be just as efficient (if not faster) than a float, at least so far as memory reads/writes is concerned.
– Razor Storm
Aug 6 '10 at 20:26
@Razor If you work exactly as many floats as fit in a cache line, then if you used doubles instead the CPU will have to fetch two cache lines. The caching effect I had in mind when reading Alex' answer however is: Your set of floats fits in you nth level cache but the corresponding set of doubles doesn't. In this case you will experience a big boost in performance if you use floats.
– Peter G.
Aug 6 '10 at 20:35
@Peter, yeah that makes sense, say you have a 32 bit cacheline, using doubles would have to fetch twice every time.
– Razor Storm
Aug 6 '10 at 20:36
@Razor, the problem's not really with fetching/storing just one value -- it is, as @Peter's focus correctly indicates, that often you're fetching "several" values to operate on (an array of numbers would be a typical example, and operations on items of such arrays very common in numerical applications). There are counterexamples (e.g., a pointer-connected tree where each node only has one number and a lot of other stuff: then having that number be 4 or 8 bytes will matter pretty little), which is part of why I say that in the end you have to benchmark, but the idea often applies.
– Alex Martelli
Aug 6 '10 at 20:55
@me pity I cannot edit the spelling mistakes of my earlier comment ...
– Peter G.
Aug 6 '10 at 21:07
If all floating-point calculations are performed within the FPU, then, no, there is no difference between a double calculation and a float calculation because the floating point operations are actually performed with 80 bits of precision in the FPU stack. Entries of the FPU stack are rounded as appropriate to convert the 80-bit floating point format to the double or float floating-point format. Moving sizeof(double) bytes to/from RAM versus sizeof(float) bytes is the only difference in speed.
double
float
double
float
sizeof(double)
sizeof(float)
If, however, you have a vectorizable computation, then you can use the SSE extensions to run four float calculations in the same time as two double calculations. Therefore, clever use of the SSE instructions and the XMM registers can allow higher throughput on calculations that only use floats.
float
double
float
Another point to consider is if you are using GPU(the graphics card). I work with a project that is numerically intensive, yet we do not need the percision that double offers. We use GPU cards to help further speed the processing. CUDA GPU's need a special package to support double, and the amount of local RAM on a GPU is quite fast, but quite scarce. As a result using float also doubles the amount of data we can store on the
Yet another point is memory. Floats take half as much RAM as doubles. If you are dealing with VERY large datasets, this can be a real important factor. If using double means you have to cache to disk vs pure ram, your difference will be huge.
So for the application I am working with, the difference is quite important.
In experiments of adding 3.3 for 2000000000 times, results are:
Summation time in s: 2.82 summed value: 6.71089e+07 // float
Summation time in s: 2.78585 summed value: 6.6e+09 // double
Summation time in s: 2.76812 summed value: 6.6e+09 // long double
So double is faster and default in C and C++. It's more portable and the default across all C and C++ library functions. Alos double has significantly higher precision than float.
Even Stroustrup recommends double over float:
"The exact meaning of single-, double-, and extended-precision is implementation-defined. Choosing the right precision for a problem where the choice matters requires significant understanding of floating-point computation. If you don't have that understanding, get advice, take the time to learn, or use double and hope for the best."
Perhaps the only case where you should use float instead of double is on 64bit hardware with a modern gcc. Because float is smaller; double is 8 bytes and float is 4 bytes.
+1 for making the effort to do some timings. But Stroustrup doesn't recommend using 'double' because it's faster, but because of the extra precision. Regarding your last comment, if you need that extra precision more than saving memory, then it's quite possible you'd want to use 'double' on 32-bit hardware. And that leads back to the question: Is double faster than float even on 32-bit hardware with a modern FPU that does 64-bit computations?
– Brent Faust
Aug 21 '12 at 17:05
A few hundredths of a second difference feels like it's still within the realm of experimental error. Especially if there's other stuff too (like maybe a not-unrolled loop . . .).
– imallett
Jul 23 '15 at 1:55
It's quite a stretch to say that Stroustrup is recommending
double there when he is actually recommending to RTFM.– sunside
Feb 27 '16 at 12:01
double
What hardware, what compiler + options, what code? If you timed all 3 in the same program, clock-speed ramp-up time explains the first being slower. Clearly you didn't enable auto-vectorization (impossible for a reduction without -ffast-math or whatever, because FP math isn't strictly associative). So this only proves that there's no speed difference when the bottleneck is scalar FP add latency. The bit about 64-bit hardware makes no sense either: float is always half the size of double on any normal hardware. The only difference on 64-bit hardware is that x86-64 has SSE2 as a baseline.
– Peter Cordes
Aug 19 '16 at 17:39
I just want to add to the already existing great answers that the __m256? family of same-instruction-multiple-data (SIMD) C++ intrinsic functions operate on either 4 double s in parallel (e.g. _mm256_add_pd), or 8 floats in parallel (e.g. _mm256_add_ps).
__m256?
double
_mm256_add_pd
float
_mm256_add_ps
I'm not sure if this can translate to an actual speed up, but it seems possible to process 2x as many floats per instruction when SIMD is used.
The only really useful answer is: only you can tell. You need to benchmark your scenarios. Small changes in instruction and memory patterns could have a significant impact.
It will certainly matter if you are using the FPU or SSE type hardware (former does all its work with 80bit extended precision, so double will be closer; later is natively 32bit, i.e. float).
Update: s/MMX/SSE/ as noted in another answer.
Floating point is normally an extension to one's general purpose CPU. The speed will therefore be dependent on the hardware platform used. If the platform has floating point support, I will be surprised if there is any difference.
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
It depends what you are doing with them. In theory, memory bandwidth could come into it. Do you have any more information?
– Kieren Johnstone
Aug 6 '10 at 17:28