Why don't C++ compilers do better constant folding?

I'm investigating ways to speed up a large section of C++ code, which has automatic derivatives for computing jacobians. This involves doing some amount of work in the actual residuals, but the majority of the work (based on profiled execution time) is in calculating the jacobians.

This surprised me, since most of the jacobians are propagated forward from 0s and 1s, so the amount of work should be 2-4x the function, not 10-12x. In order to model what a large amount of the jacobian work is like, I made a super minimal example with just a dot product (instead of sin, cos, sqrt and more that would be in a real situation) that the compiler should be able to optimize to a single return value:

#include <Eigen/Core>
#include <Eigen/Geometry>

using Array12d = Eigen::Matrix<double,12,1>;

double testReturnFirstDot(const Array12d& b)

Array12d a;
a.array() = 0.;
a(0) = 1.;
return a.dot(b);



Which should be the same as

double testReturnFirst(const Array12d& b)

return b(0);



I was disappointed to find that, without fast-math enabled, neither GCC 8.2, Clang 6 or MSVC 19 were able to make any optimizations at all over the naive dot-product with a matrix full of 0s. Even with fast-math (https://godbolt.org/z/GvPXFy) the optimizations are very poor in GCC and Clang (still involve multiplications and additions), and MSVC doesn't do any optimizations at all.

I don't have a background in compilers, but is there a reason for this? I'm fairly sure that in a large proportion of scientific computations being able to do better constant propagation/folding would make more optimizations apparent, even if the constant-fold itself didn't result in a speedup.

While I'm interested in explanations for why this isn't done on the compiler side, I'm also interested for what I can do on a practical side to make my own code faster when facing these kinds of patterns.

(1.0 / 3.0) * 3.0

(1.0 * 3.0)/3.0

dot

for

-ffast-math

/fp:fast

-ffast-math

3 Answers
3

This is because Eigen explicitly vectorize your code as 3 vmulpd, 2 vaddpd and 1 horizontal reduction within the remaining 4 component registers (this assumes AVX, with SSE only you'll get 6 mulpd and 5 addpd). With -ffast-math GCC and clang are allowed to remove the last 2 vmulpd and vaddpd (and this is what they do) but they cannot really replace the remaining vmulpd and horizontal reduction that have been explicitly generated by Eigen.

-ffast-math

So what if you disable Eigen's explicit vectorization by defining EIGEN_DONT_VECTORIZE? Then you get what you expected (https://godbolt.org/z/UQsoeH) but other pieces of code might become much slower.

EIGEN_DONT_VECTORIZE

If you want to locally disable explicit vectorization and are not afraid of messing with Eigen's internal, you can introduce a DontVectorize option to Matrix and disable vectorization by specializing traits<> for this Matrix type:

DontVectorize

Matrix

traits<>

Matrix

static const int DontVectorize = 0x80000000;

namespace Eigen
namespace internal

template<typename _Scalar, int _Rows, int _Cols, int _MaxRows, int _MaxCols>
struct traits<Matrix<_Scalar, _Rows, _Cols, DontVectorize, _MaxRows, _MaxCols> >
: traits<Matrix<_Scalar, _Rows, _Cols> >

typedef traits<Matrix<_Scalar, _Rows, _Cols> > Base;
enum
EvaluatorFlags = Base::EvaluatorFlags & ~PacketAccessBit
;
;




using ArrayS12d = Eigen::Matrix<double,12,1,DontVectorize>;


Full example there: https://godbolt.org/z/bOEyzv

I was disappointed to find that, without fast-math enabled, neither GCC 8.2, Clang 6 or MSVC 19 were able to make any optimizations at all over the naive dot-product with a matrix full of 0s.

They have no other choice unfortunately. Since IEEE floats have signed zeros, adding 0.0 is not an identity operation:

0.0

-0.0 + 0.0 = 0.0 // Not -0.0!


Similarly, multiplying by zero does not always yield zero:

0.0 * Infinity = NaN // Not 0.0!


So the compilers simply cannot perform these constant folds in the dot product while retaining IEEE float compliance - for all they know, your input might contain signed zeros and/or infinities.

You will have to use -ffast-math to get these folds, but that may have undesired consequences. You can get more fine-grained control with specific flags (from http://gcc.gnu.org/wiki/FloatingPointMath). According to the above explanation, adding the following two flags should allow the constant folding:
-ffinite-math-only, -fno-signed-zeros

-ffast-math

-ffinite-math-only

-fno-signed-zeros

Indeed, you get the same assembly as with -ffast-math this way: https://godbolt.org/z/vGULLA. You only give up the signed zeros (probably irrelevant), NaNs and the infinities. Presumably, if you were to still produce them in your code, you would get undefined behavior, so weigh your options.

-ffast-math

As for why your example is not optimized better even with -ffast-math: That is on Eigen. Presumably they have vectorization on their matrix operations, which are much harder for compilers to see through. A simple loop is properly optimized with these options: https://godbolt.org/z/OppEhY

-ffast-math

for

One way to force a compiler to optimize multiplications by 0's and 1`s is to manually unroll the loop. For simplicity let's use

#include <array>
#include <cstddef>
constexpr std::size_t n = 12;
using Array = std::array<double, n>;


Then we can implement a simple dot function using fold expressions (or recursion if they are not available):

dot

<utility>
template<std::size_t... is>
double dot(const Array& x, const Array& y, std::index_sequence<is...>)

return ((x[is] * y[is]) + ...);


double dot(const Array& x, const Array& y)

return dot(x, y, std::make_index_sequence<n>);



Now let's take a look at your function

double test(const Array& b)

const Array a1; // = 1, 0, ...
return dot(a, b);



With -ffast-math gcc 8.2 produces:

-ffast-math

test(std::array<double, 12ul> const&):
movsd xmm0, QWORD PTR [rdi]
ret


clang 6.0.0 goes along the same lines:

test(std::array<double, 12ul> const&): # @test(std::array<double, 12ul> const&)
movsd xmm0, qword ptr [rdi] # xmm0 = mem[0],zero
ret


For example, for

double test(const Array& b)

const Array a1, 1; // = 1, 1, 0...
return dot(a, b);



we get

test(std::array<double, 12ul> const&):
movsd xmm0, QWORD PTR [rdi]
addsd xmm0, QWORD PTR [rdi+8]
ret


Addition. Clang unrolls a for (std::size_t i = 0; i < n; ++i) ... loop without all these fold expressions tricks, gcc doesn't and needs some help.

for (std::size_t i = 0; i < n; ++i) ...

Thanks for contributing an answer to Stack Overflow!

But avoid …

To learn more, see our tips on writing great answers.

Some of your past answers have not been well-received, and you're in danger of being blocked from answering.

Please pay close attention to the following guidance:

But avoid …

To learn more, see our tips on writing great answers.

Required, but never shown

Required, but never shown

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.