MATLAB® R11.1 Bug Fix

Although Signalogic engineers ended up doing the work to implement the fix for this bug, The MathWorks technical support personnel were very helpful and basically pointed right at the problem. Thanks very much TMW. TMW really has very good technical support, especially when considering the large number of scientific and research users of their products who are likely to ask every possible question, in the smallest detail.

Especially Peter Torrione was very helpful and polite, even though Mr. Brower at Signalogic was not always in a good mood in his e-mails.

From: Jeff Brower (jbrower@signalogic.com)
Subject: new problem with MATLAB calling external DLL functions in R11.1
To: Shannon
Cc: Jeff, Cheryl, V.,  Yolanda (yolanda@signalogic.com)
Date: 30 Jan, 2000

Shannon-

R11.1 has a bug in calling library functions in an external DLL previously
compiled for R11.

MathWorks developers have (again) either a) updated to a new compiler version
or b) changed the way they call exported mexFunctions from external DLLs (e.g.
calling convention, number of parameters on the stack, who pops the stack when
the function is done, etc).  This means that a DLL developed for R11 now
crashes with R11.1, unless it is re-compiled.

We have tracked this down and we know what to change in our DLLs to work
around and return to compatibility, but the problem is we are a MathWorks
3rd-party, and we have over 330 customers using our DirectDSP and DSPower
software with MATLAB.  The Signalogic software includes DLLs and ActiveX
components which are called by .m functions.  We would like to avoid shipping
separate sets of DLLs to MATLAB customers depending on their MATLAB version;
this would be a logistical mess.

Two questions:

1) Is it possible for your developers to maintain a standard test DLL, in
executable form, and ensure it functions WITHOUT CHANGES from one version of
MATLAB to the next?  MathWorks is forcing previous versions of Win32
components to change when the application changes.  This violates one of the
fundamental reasons why Windows supports components, and in fact does not
meet Microsoft application certification standards.

2) How can a 3rd-party like Signalogic get involved in pre-release and/or beta
programs, so we can see new releases BEFORE they hit the street, to give us a
chance to find this type of bug and bring your attention to it?  I realize
that 330+ customers is a small drop for MathWorks, but it is not
insignificant, and we do provide value for MATLAB customers who are using
DSP/data acquisition boards from Texas Instruments, Motorola, Analog Devices,
and other DSP vendors.

Jeff Brower
Signalogic





To: jeff@signalogic.com
From: Peter
Subject: Re: Case ID: 346528 questions - API
Date: Tue, 01 Feb 2000 16:10:55 -0500

This is in response to Case ID:  346528

Dear Jeff,

I am writing in response to your e-mail regarding MATLAB Programming.

>1) Is it possible for your developers to maintain a standard
>test DLL, in  executable form, and ensure it functions
>WITHOUT CHANGES from one version of MATLAB to the next?

Of course the MathWorks tries its best to keep code maintenance to a
minimum, including the need to recompile old programs.  Of course, often
bug fixes and improvements have to be made that alter the way a program
works.  The MATLAB API acts as more than just a program, however.  It is an
interface that many different languages and programs utilize.

>2) How can a 3rd-party like Signalogic get involved in
>pre-release and/or beta programs, so we can see new releases
>BEFORE they hit the street, to give us a  chance to find
>this type of bug and bring your attention to it?

Judging from your e-mail I would like to nominate you for inclusion on our
R12 Beta 4 mailing list.

Can you please send me your full mailing address?

Thank you.

Sincerely,

Peter
Technical Support Engineer





From: Jeff Brower (jbrower@signalogic.com)
Subject: Re: Case ID: 346528 questions - API
To: Peter
Cc: Loren, Dr. Roy, Jeff, Cheryl, V., Yolanda
	(yolanda@signalogic.com)
Date: 5 Feb, 2000

Peter-

Thanks for your fast reply.

>>1) Is it possible for your developers to maintain a standard
>>test DLL, in  executable form, and ensure it functions
>>WITHOUT CHANGES from one version of MATLAB to the next?
>
>Of course the MathWorks tries its best to keep code maintenance to a
>minimum, including the need to recompile old programs.  Of course, often
>bug fixes and improvements have to be made that alter the way a program
>works.  The MATLAB API acts as more than just a program, however.  It is an
>interface that many different languages and programs utilize.

Peter, I'm not talking about .m files.  I'm talking about external DLLs --
ActiveX and other Windows components.  Who can I talk to about this?  Loren or
Roy, can you help us here?  I want to get this taken care of directly with
MathWorks rather than post it on news groups.  There needs to be a solution;
it is not reasonable for new releases of MATLAB to fail with previously
compiled DLLs.

>Judging from your e-mail I would like to nominate you for inclusion on our
>R12 Beta 4 mailing list.
>
>Can you please send me your full mailing address?

Thanks very much; it is included below after copy of previous e-mail.

Jeff Brower
Signalogic





To: jeff@signalogic.com
From: Peter
Subject: Re: Case ID: 346528 questions - API
Cc: Cheryl
Date: Mon, 07 Feb 2000 12:18:39 -0500

This is in response to Case ID:  346528

Dear Jeff,

I am writing in response to your e-mail regarding using MATLAB.

> The MATLAB API acts as more than just a program, however.
>It is an interface that many different languages and
>programs utilize.
>Peter, I'm not talking about .m files.  I'm talking about
>external DLLs -- ActiveX and other Windows components.
...
>There needs to be a solution; it is not reasonable for
>new releases of MATLAB to fail with previously compiled
>DLLs.

We are not aware of any incompatibilities between release 11 and Release
11.1 for MATLAB DLL files.

In general when there will be a need to recompile a MATLAB program, we are
aware of it at shipping time and we will include a note to that effect in
the Product's Release Notes.

If possible, I would like to get a copy of the MEX file that causes this
problem when run on MATLAB 5.3.1 to see if we overlooked something in the
upgrade.  Please send me the Original C files and the MEX command you
issued in MATLAB 5.3.  Also, please let me know what C Compiler you are
using, the operating system you are on, and how to run your program.

I have also entered you to be included in future MATLAB Beta testing.

Sincerely,

Peter
Technical Support Engineer






From: Jeff Brower (jbrower@signalogic.com)
Subject: Re: Case ID: 346528 questions - API
To: Peter
Cc: Loren, Dr. Roy, Shannon, Jeff, Cheryl, V.,
Yolanda (yolanda@signalogic.com)
Date: 6 Mar, 2000

Peter-

>We are not aware of any incompatibilities between release 11 and Release
>11.1 for MATLAB DLL files.

I have attached two (2) zip files:

  mtest_52.zip
  mtest_53.zip

Here are some notes about the attached files:

  -bugtest.m calls calldll.m which calls the
   mtest.dll; all files are completely
   minimized and very short; an ultra-simple
   example

  -mtest.cpp is the source code for the DLL;
   again, it is ultra-simple and just does a
   message box which demonstrates that the
   MATLAB crash occurs on DLL return, after
   the DLL exits the mexFunction()

  -the only differences in _53.zip and _52.zip
   are the mtest.cpp file (two possible
   mexFunction() declaration lines flipped;
   either one commented-out or the other) and
   different mtest.dll files.  Suggest not to
   unzip on top of each other since filenames
   are the same.

The mtest.dll file included in mtest_52.zip can be run by MATLAB versions v5.2
through v5.3 R11, but fails with v5.3 R11.1 (GPF hard crash).  The mtest.dll
file included in mtest_53.zip  can be run by v5.3 R11 and R11.1, but fails
with anything earlier.

So basically you have a mess.  DLL files created by 3rd-parties can no longer
be distributed to MATLAB customers without regard to WHICH v5.x version the
customer has.

The only difference in mtest.cpp that causes the incompatibility among
versions is the presence or absence of the "WINAPI" attribute in the
mexFunction() declaration.  Based on this observation, my guess is that
somewhere in changing compilers, MathWorks has also changed the parameter
passing convention.  It appears that now TMW has ceased to use the standard
Windows "Pascal" format for exported DLL functions, where parameters are
pushed in order, and have instead switched to "C" reversed parameter
order.  It may seem TMW has no need to be compatible with standard Microsoft
DLL function calls, since you expect only MATLAB to call DLL functions, and
not other Win programs.  But on the other hand, you also claim Microsoft
compatibility and display the approved Microsoft logos.  Would Microsoft
still grant this approval if they knew that TMW DLLs are non-standard?

>In general when there will be a need to recompile a MATLAB program, we are
>aware of it at shipping time and we will include a note to that effect in
>the Product's Release Notes.

I guess not this time.

>If possible, I would like to get a copy of the MEX file that causes this
>problem when run on MATLAB 5.3.1 to see if we overlooked something in the
>upgrade.  Please send me the Original C files and the MEX command you
>issued in MATLAB 5.3.  Also, please let me know what C Compiler you are
>using, the operating system you are on, and how to run your program.

See above.

>I have also entered you to be included in future MATLAB Beta testing.

Thanks very much!  This well help us a lot in avoiding surprises where our
MATLAB customers detect problems before we do.

Jeff Brower
Signalogic
((Attachment: MTEST_52.ZIP))
((Attachment: MTEST_53.ZIP))





To: jeff@signalogic.com
From: Peter
Subject: Re: Case ID: 364161 Crash in ML 5.3.1
Date: Tue, 14 Mar 2000 16:50:21 -0500

This is in response to Case ID:  364161

Dear Jeff,

I am writing in response to your e-mail regarding using MATLAB MEX files in
R11 and R11.1 (5.3 and 5.3.1).

>The mtest.dll file included in mtest_52.zip can be run by
>MATLAB versions v5.2 through v5.3 R11, but fails with v5.3
>R11.1 (GPF hard crash).  The mtest.dll file included in
>mtest_53.zip  can be run by v5.3 R11 and R11.1, but fails
>with anything earlier.

I was able to reproduce these results using the DLL files that you shipped
to me.  It appears that in this case, a small correction in the way we
handled the parameter passing stack has caused this problem.  However,
MATLAB MEX files have always assumed the same calling conventions.  If
MATLAB finds the mexFunction entry point in a DLL, it automatically assumes
to use C calling conventions (caller removes variables from the stack).  In
MATLAB 5.3, it appears that sometimes this was improperly done, so making a
function follow WINAPI calling techniques did not cause a segmentation fault.

In MATLAB R11.1 we correctly clear the stack for every call.  This results
in the errors you were seeing.

Also, I was able to get the MATLAB 5.3.1 calling technique to work in both
5.3 and 5.3.1:

void _export mexFunction(int nlhs, mxArray* plhs[], int nrhs, const
mxArray* prhs[]) {
   mexPrintf("Version 5.3.1 (R11.1) calling convention\n");
}

%%%%%%%%%%%
» mex mtest53.cpp
» mtest53
Version 5.3.1 (R11.1) calling convention
» version
ans =
5.3.0.10183 (R11)
%%%%%%%%%%%

%%%%%%%%%%%%
» mex mtest531.cpp
» mtest531
Version 5.3.1 (R11.1) calling convention
» version
ans =
5.3.1.29215a (R11.1)
%%%%%%%%%%%%


Also, I was unable to compile the MEX files that you sent to me when I
included "WINAPI" in the declaration of the MEX files that you sent:

>> mex mtest531.cpp
(SNIP)
mtest531.cpp(56) : error C2373: 'mexFunction' : redefinition; different
type modifiers
 D:\MATLABR11_1\extern\include\mex.h(168) : see declaration of 'mexFunction'

> But on the other hand, you also claim Microsoft
>compatibility and display the approved Microsoft
>logos.  Would Microsoft still grant this approval
>if they knew that TMW DLLs are non-standard?

You are correct, MATLAB MEX files are designed to only be called from
within MATLAB.  No other program con realistically expect to call a MEX
file (mex files can actually make calls to the MATLAB interpreter /
engine).  Although MEX files cannot be compiled with WINAPI, the *external*
DLL's that MATLAB generates (with the Compiler / Math and Graphics
Libraries) for external usage are different, and should behave like normal
Windows DLL's.

I apologize for any inconvenience this undocumented change between versions
may have caused you.

If you have any more questions regarding this matter, feel free to e-mail
me.

Sincerely,

Peter
Technical Support Engineer





From: Jeff Brower (jbrower@signalogic.com)
Subject: Re: Case ID: 364161 Crash in ML 5.3.1
To: Peter
Cc: Loren, Dr. Roy, Shannon, Jeff, Cheryl, V.,
Yolanda (yolanda@signalogic.com), Tam (tam@signalogic.com)
Date: 16 Mar, 2000

Peter-

>I am writing in response to your e-mail regarding using MATLAB MEX files in
>R11 and R11.1 (5.3 and 5.3.1).

Thanks very much for your reply.

>>The mtest.dll file included in mtest_52.zip can be run by
>>MATLAB versions v5.2 through v5.3 R11, but fails with v5.3
>>R11.1 (GPF hard crash).  The mtest.dll file included in
>>mtest_53.zip  can be run by v5.3 R11 and R11.1, but fails
>>with anything earlier.
>
>I was able to reproduce these results using the DLL files that you shipped
>to me.  It appears that in this case, a small correction in the way we
>handled the parameter passing stack has caused this problem.  However,
>MATLAB MEX files have always assumed the same calling conventions.  If
>MATLAB finds the mexFunction entry point in a DLL, it automatically assumes
>to use C calling conventions (caller removes variables from the stack).  In
>MATLAB 5.3, it appears that sometimes this was improperly done, so making a
>function follow WINAPI calling techniques did not cause a segmentation fault.
>
>In MATLAB R11.1 we correctly clear the stack for every call.  This results
>in the errors you were seeing.

Ok, this makes sense and would be an explanation for what we are seeing.

>Also, I was able to get the MATLAB 5.3.1 calling technique to work in both
>5.3 and 5.3.1:

Right, we were able to do this also.  But the problem remains that we cannot
get the same Mex function to work with both 5.3/5.3.1, and earlier versions.

>Also, I was unable to compile the MEX files that you sent to me when I
>included "WINAPI" in the declaration of the MEX files that you sent:
>
>>> mex mtest531.cpp
>(SNIP)
>mtest531.cpp(56) : error C2373: 'mexFunction' : redefinition; different
>type modifiers
> D:\MATLABR11_1\extern\include\mex.h(168) : see declaration of 'mexFunction'

You will need to use an earlier version of mex.h; i.e. one for 5.3 or
earlier.  In the example project files we sent, the reference (location)
changes for the mex.h file for the two projects, but this is not obvious in
the files we sent and is specific to our test system.

>> But on the other hand, you also claim Microsoft
>>compatibility and display the approved Microsoft
>>logos.  Would Microsoft still grant this approval
>>if they knew that TMW DLLs are non-standard?
>
>You are correct, MATLAB MEX files are designed to only be called from
>within MATLAB.  No other program con realistically expect to call a MEX
>file (mex files can actually make calls to the MATLAB interpreter /
>engine).

Of course nothing but MATLAB wants to call a Mex function, but what if the Mex
function is embedded in a standard product, i.e a DLL, OCX, or other ActiveX
component with other, standard Win-callable functions?  MathWorks still has
the problem of requiring people to make a non-standard DLL, thus violating
Microsoft approval guidelines and invalidating the Microsoft logo published
with MathWorks software.

>I apologize for any inconvenience this undocumented change between versions
>may have caused you.

Well, we still don't have an answer or work-around.  We are still in the
position of having to ship multiple versions of the same product, with
*slight* and incompatible internal differences, to support different versions
of MATLAB.  And we have add performance-degrading code to our .m toolboxes to
make sure the correct library calls are made, depending on which MATLAB
version the code is running under.

Thanks very much Peter for your efforts.  I think you have found the cause of
the problem.  What we will do now is implement a work-around to the problem
that automatically recognizes when the stack should be cleared by our Mex
functions prior to returning to MATLAB and when not.  I hope this type of
dynamic work-around is possible because it would keep us out of a software
distribution logistics mess where our MATLAB 3rd-party products are concerned.

Jeff Brower
Signalogic