Is there some problem you are running into?

%octave_pkg_build defines the macro in rpm, so it is available later any other use.

Yes. Here in mathgl %octave_pkg_build is not called. Bindings are built with cmake itself from swig .i files. But installation of octave module is done in a way of tar file copied in somewhere only. No registration, no debuginfo extraction, no stripping (what results in a *very* big file size). So to use install macro here looks like not so bad idea.

It worked in 4.0 and it fails in 4.2.

If it's mathgl-related only may be I should define this var in mathgl.spec in some kind of %global ? That could be a solution I suppose.

(if it really triggers *only* mathgl)

Well, it like is only mathgl that call %octave_pkg_install without calling %octave_pkg_build.  A workaround would be to do

%global octave_tar_suffix %{octave_host}-%{octave_api}

It's actually a tricky problem to solve because it's conditional on whether or not the package is noarch or not, which is easiest to do in the %octave_pkg_build lua macro.  I may convert %octave_pkg_install to lua, but that will take a little time.

Well. That's not so simple. There are too much differences in install scripts between 4.0 and 4.2. Have to handle them by hands.
OK, I'l try.

Here is a part of spec:

%if 0%{?fedora} >= 21 && 0%{?fedora} < 26
%global mgl_octarch_dir %{_tmppath}/%{name}-%{version}-%{release}.%{_arch}
%global mgl_octarch_name %{octpkg}-%{version}.tar.gz
%endif
%if 0%{?fedora} >= 26
%global octave_tar_suffix %{octave_host}-%{octave_api}
%global mgl_octarch_dir %{_builddir}/%{buildsubdir}/build/
%global mgl_octarch_name %{octpkg}-%{version}-%{octave_tar_suffix}.tar.gz
%endif

...

%if 0%{?with_octave}
mkdir -p %{mgl_octarch_dir}
cp %{_target_platform}_serial/lang/%{octpkg}.tar.gz %{mgl_octarch_dir}/%{mgl_octarch_name}
%octave_pkg_install
%endif

And it works.
So, if %octave_pkg_install will be dramatically changed ping me please.