[dcl.init.ref] subclause 5.4.1 says:

If T1 or T2 is a class type and T1 is not reference-related to T2, user-defined conversions are considered using the rules for copy-initialization of an object of type “cv1 T1” by user-defined conversion; the program is ill-formed if the corresponding non-reference copy-initialization would be ill-formed. The result of the call to the conversion function, as described for the non-reference copy-initialization, is then used to direct-initialize the reference. For this direct-initialization, user-defined conversions are not considered.

It states that used-defined conversions are considered (using the rules for copy-initialization of an object), however, it only specifies what will happen when a conversion function is chosen, and there is no mention of what would happen if a constructor is chosen.