Bug 1215630

If you get a compiler error, can you please:
1) say what exact error do you get (copy & paste from the gcc output)
2) say exact gcc command line options
3) attach preprocessed source that reproduces it?

Sorry, I've attempt to report it to upstream at the same time and missing those basics informations:

That should be for 1/ & 2/, 3/ will follow

../doltlibtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -DMODULE_STRING=\"$(p="video_chroma/libi420_yuy2_sse2_plugin_la-i420_yuy2.lo"; p="${p##*/}"; p="${p#lib}"; p="${p%_plugin*}"; p="${p%.lo}"; echo "$p")\" -D__PLUGIN__  -I./access -I./codec -DMODULE_NAME_IS_i420_yuy2_sse2 -I../include -I../include   -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -Wall -Wextra -Wsign-compare -Wundef -Wpointer-arith -Wbad-function-cast -Wwrite-strings -Wmissing-prototypes -Wvolatile-register-var -Werror-implicit-function-declaration -pipe -fvisibility=hidden -O4 -ffast-math -funroll-loops -fomit-frame-pointer -c -o video_chroma/libi420_yuy2_sse2_plugin_la-i420_yuy2.lo `test -f 'video_chroma/i420_yuy2.c' || echo './'`video_chroma/i420_yuy2.c
In file included from video_chroma/i420_yuy2.c:35:0:
video_chroma/i420_yuy2.c: In function 'I420_UYVY_Filter':
video_chroma/i420_yuy2.c:194:23: error: calling 'I420_UYVY.isra.2' with SSE caling convention without SSE/SSE2 enabled
 VIDEO_FILTER_WRAPPER( I420_UYVY )
                       ^
../include/vlc_filter.h:266:13: note: in definition of macro 'VIDEO_FILTER_WRAPPER'
             name( p_filter, p_pic, p_outpic );                          \
             ^
video_chroma/i420_yuy2.c: In function 'I420_YUY2_Filter':
video_chroma/i420_yuy2.c:192:23: error: calling 'I420_YUY2.isra.0' with SSE caling convention without SSE/SSE2 enabled
 VIDEO_FILTER_WRAPPER( I420_YUY2 )
                       ^
../include/vlc_filter.h:266:13: note: in definition of macro 'VIDEO_FILTER_WRAPPER'
             name( p_filter, p_pic, p_outpic );                          \
             ^
video_chroma/i420_yuy2.c: In function 'I420_cyuv_Filter':
video_chroma/i420_yuy2.c:197:23: error: calling 'I420_cyuv.isra.3' with SSE caling convention without SSE/SSE2 enabled
 VIDEO_FILTER_WRAPPER( I420_cyuv )
                       ^
../include/vlc_filter.h:266:13: note: in definition of macro 'VIDEO_FILTER_WRAPPER'
             name( p_filter, p_pic, p_outpic );                          \
             ^
video_chroma/i420_yuy2.c: In function 'I420_YVYU_Filter':
video_chroma/i420_yuy2.c:193:23: error: calling 'I420_YVYU.isra.1' with SSE caling convention without SSE/SSE2 enabled
 VIDEO_FILTER_WRAPPER( I420_YVYU )
                       ^
../include/vlc_filter.h:266:13: note: in definition of macro 'VIDEO_FILTER_WRAPPER'
             name( p_filter, p_pic, p_outpic );                          \
             ^
Makefile:8137: recipe for target 'video_chroma/libi420_yuy2_sse2_plugin_la-i420_yuy2.lo' failed
make[4]: *** [video_chroma/libi420_yuy2_sse2_plugin_la-i420_yuy2.lo] Error 1
make[4]: Leaving directory '/builddir/build/BUILD/vlc-2.2.1/modules'
Makefile:9651: recipe for target 'all-recursive' failed
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory '/builddir/build/BUILD/vlc-2.2.1/modules'
Makefile:4525: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory '/builddir/build/BUILD/vlc-2.2.1/modules'
Makefile:2261: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/builddir/build/BUILD/vlc-2.2.1'
Makefile:2146: recipe for target 'all' failed
make: *** [all] Error 2

Created attachment 1019760 [details]
compressed build.log from mock for vlc-2.2.1-2.fc22 on i686

Created attachment 1019762 [details]
gcc -E output from the failing file

I've disabled optimizations (which removes -O4 -ffast-math -funroll-loops -fomit-frame-pointer and -Wl,-Bsymbolic as ldflag), but the problem is the same.

Dunno how you've preprocessed the file, but there are 3 occurrences of VLC_TARGET
unexpanded in the source, like:
# 199 "video_chroma/i420_yuy2.c"
VLC_TARGET
static void I420_YUY2( filter_t *p_filter, picture_t *p_source,
                                           picture_t *p_dest )
{
which make it of course invalid C.  If I remove the VLC_TARGET lines, everything compiles just fine.

I hit the same error trying to compile vlc at Linux Arch and looking for a solution.

The build script is here https://projects.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/vlc

The compiler output:

make[4]: *** [video_chroma/libi422_yuy2_sse2_plugin_la-i422_yuy2.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
In file included from video_chroma/i420_yuy2.c:35:0:
video_chroma/i420_yuy2.c: In function 'I420_UYVY_Filter':
video_chroma/i420_yuy2.c:194:23: error: calling 'I420_UYVY.isra.2' with SSE caling convention without SSE/SSE2 enabled
 VIDEO_FILTER_WRAPPER( I420_UYVY )
                       ^
../include/vlc_filter.h:266:13: note: in definition of macro 'VIDEO_FILTER_WRAPPER'
             name( p_filter, p_pic, p_outpic );                          \
             ^
video_chroma/i420_yuy2.c: In function 'I420_YUY2_Filter':
video_chroma/i420_yuy2.c:192:23: error: calling 'I420_YUY2.isra.0' with SSE caling convention without SSE/SSE2 enabled
 VIDEO_FILTER_WRAPPER( I420_YUY2 )
                       ^
../include/vlc_filter.h:266:13: note: in definition of macro 'VIDEO_FILTER_WRAPPER'
             name( p_filter, p_pic, p_outpic );                          \
             ^
video_chroma/i420_yuy2.c: In function 'I420_cyuv_Filter':
video_chroma/i420_yuy2.c:197:23: error: calling 'I420_cyuv.isra.3' with SSE caling convention without SSE/SSE2 enabled
 VIDEO_FILTER_WRAPPER( I420_cyuv )
                       ^
../include/vlc_filter.h:266:13: note: in definition of macro 'VIDEO_FILTER_WRAPPER'
             name( p_filter, p_pic, p_outpic );                          \
             ^
video_chroma/i420_yuy2.c: In function 'I420_YVYU_Filter':
video_chroma/i420_yuy2.c:193:23: error: calling 'I420_YVYU.isra.1' with SSE caling convention without SSE/SSE2 enabled
 VIDEO_FILTER_WRAPPER( I420_YVYU )
                       ^
../include/vlc_filter.h:266:13: note: in definition of macro 'VIDEO_FILTER_WRAPPER'
             name( p_filter, p_pic, p_outpic );                          \
             ^
Makefile:8134: recipe for target 'video_chroma/libi420_yuy2_sse2_plugin_la-i420_yuy2.lo' failed
make[4]: *** [video_chroma/libi420_yuy2_sse2_plugin_la-i420_yuy2.lo] Error 1

The error I posted happens if try to build VLC with gcc 5.1.0. But if I build VLC with gcc-4.9.2 it looks fine. Can anyone confirm the same behavior?

I can confirm that this issue occurs on i686 though I'm not sure if it's a GCC or VLC bug.

Below is a small test case based on what VLC is doing.

File: sse-target-test.c
============================
__attribute__ ((__target__ ("sse")))
static void foo()
{
    __asm__ __volatile__("nop");
};

void foo_wrapper () { foo(); }
============================

The following will succeed:

$ gcc -c sse-target-test.c
$ gcc -c sse-target-test.c -O1 -msse

But this will result in an error:

$ gcc -c sse-target-test.c -O1 
sse-target-test.c: In function ‘foo_wrapper’:
sse-target-test.c:7:23: error: calling ‘foo’ with SSE caling convention without SSE/SSE2 enabled
 void foo_wrapper () { foo(); }

Tracking this upstream now.

Short term workaround, stick __attribute__((used)) on the problematic functions with target ("sse") (or sse2) attribute that are called from functions compiled without that ISA, that will force GCC not to attempt to change the local calling convention.