Discussion:
[Cocci] [cocci-bug] coccinelle 1.0.6: OpenMP #pragma directive leads to function ignore
Julia Lawall
2018-04-26 08:25:54 UTC
Permalink
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it.
If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.
Is there anything I can do to restructure large codes containing
many of such similar lines ?
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more
flexible.

julia
Julia Lawall
2018-04-26 10:22:46 UTC
Permalink
Post by Julia Lawall
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it.
If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.
Is there anything I can do to restructure large codes containing
many of such similar lines ?
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more
flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to
e.g. ignore pragma lines as they were comments.
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines
If there is any better solution, please let me know.
I don't see any other solution, sorry. For some other parsing issues, it
might be possible to define a macro, but that is not possible here.

julia
SF Markus Elfring
2018-04-26 13:01:23 UTC
Permalink
We can try to make the parsing of #pragmas more flexible.
I am also curious on further software evolution in this area for a while.
https://github.com/coccinelle/coccinelle/issues/51

Regards,
Markus
Julia Lawall
2018-04-30 06:43:24 UTC
Permalink
Post by Julia Lawall
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it.
If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.
Is there anything I can do to restructure large codes containing
many of such similar lines ?
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more
flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to
e.g. ignore pragma lines as they were comments.
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines
If there is any better solution, please let me know.
The problem is now fixed in the github version of Coccinelle.

julia
Michele Martone
2018-04-30 07:39:30 UTC
Permalink
Post by Julia Lawall
Post by Julia Lawall
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it.
If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.
Is there anything I can do to restructure large codes containing
many of such similar lines ?
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more
flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to
e.g. ignore pragma lines as they were comments.
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines
If there is any better solution, please let me know.
The problem is now fixed in the github version of Coccinelle.
julia
That is good news -- thank you :-)

Handling such pragma lines opens new ranges of use for Coccinelle!

On my side, I just followed install.txt:

#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error
git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here
# make install

and it reaches:

...
OCAMLOPT enter.ml
OCAMLC main.mli
OCAMLOPT main.ml
OCAMLOPT -o spatch.opt
/usr/lib64/ocaml/libbigarray.a(bigarray_stubs.o): In function `caml_ba_deserialize':
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to `caml_umul_overflow'
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to `caml_umul_overflow'
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read

It's my first experience building any ocaml-based software, so
I would appreciate any hint how to proceed further, if possible.

Michele
Julia Lawall
2018-04-30 07:42:42 UTC
Permalink
Post by Michele Martone
Post by Julia Lawall
Post by Julia Lawall
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it.
If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.
Is there anything I can do to restructure large codes containing
many of such similar lines ?
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more
flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to
e.g. ignore pragma lines as they were comments.
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines
If there is any better solution, please let me know.
The problem is now fixed in the github version of Coccinelle.
julia
That is good news -- thank you :-)
Handling such pragma lines opens new ranges of use for Coccinelle!
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error
git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here
# make install
...
OCAMLOPT enter.ml
OCAMLC main.mli
OCAMLOPT main.ml
OCAMLOPT -o spatch.opt
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to `caml_umul_overflow'
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to `caml_umul_overflow'
collect2: error: ld returned 1 exit status
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read
It's my first experience building any ocaml-based software, so
I would appreciate any hint how to proceed further, if possible.
Oops, I'll forward this to my engineer, who knows about building issues.
I'm not sure if he will answer before Wednesday, due to the holiday.
What is your OS?

julia
Michele Martone
2018-04-30 07:47:28 UTC
Permalink
Post by Julia Lawall
Post by Michele Martone
Post by Julia Lawall
Post by Julia Lawall
Dear Prof. Lawall and Coccinelle team,
I'm an enthusiastical, however still beginner use of Coccinelle.
While running spatch 1.0.6 on a long OpenMP-enabled listing, I
#pragma omp parallel for reduction(+:I) private(I)
seems to prevent restructuring of the function containing it.
If I simplify the pragma to something like
#pragma omp parallel
the problem disappears.
Is there anything I can do to restructure large codes containing
many of such similar lines ?
In the attachment you find a test case and the output I experience.
Thanks for the report. We can try to make the parsing of #pragmas more
flexible.
julia
Dear Julia,
thanks for the quick response!
I'll be waiting for Coccinelle-sided developments then.
May I ask what would you recommend now to circumvent this ?
From the SmPL v1.0.6 Grammar I am not aware of any way to
e.g. ignore pragma lines as they were comments.
- comment #pragma lines
- compute semantic patch
- apply semantic patch
- uncomment #pragma lines
If there is any better solution, please let me know.
The problem is now fixed in the github version of Coccinelle.
julia
That is good news -- thank you :-)
Handling such pragma lines opens new ranges of use for Coccinelle!
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error
git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here
# make install
...
OCAMLOPT enter.ml
OCAMLC main.mli
OCAMLOPT main.ml
OCAMLOPT -o spatch.opt
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to `caml_umul_overflow'
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to `caml_umul_overflow'
collect2: error: ld returned 1 exit status
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read
It's my first experience building any ocaml-based software, so
I would appreciate any hint how to proceed further, if possible.
Oops, I'll forward this to my engineer, who knows about building issues.
I'm not sure if he will answer before Wednesday, due to the holiday.
What is your OS?
$ cat /etc/os-release
NAME="openSUSE Leap"
VERSION="42.3"
ID=opensuse
ID_LIKE="suse"
VERSION_ID="42.3"
PRETTY_NAME="openSUSE Leap 42.3"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:42.3"

thanks!
Peter Senna Tschudin
2018-04-30 07:44:19 UTC
Permalink
[...]
Post by Michele Martone
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error
git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here
# make install
...
OCAMLOPT enter.ml
OCAMLC main.mli
OCAMLOPT main.ml
OCAMLOPT -o spatch.opt
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to `caml_umul_overflow'
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to `caml_umul_overflow'
collect2: error: ld returned 1 exit status
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read
I usually ask the package manager to install build dependencies for me:

Fedora:
$ sudo dnf builddep coccinelle

Ubuntu/Debian:
$ sudo apt-get build-dep coccinelle

[...]
--
Peter
Michele Martone
2018-05-02 13:16:39 UTC
Permalink
Post by Peter Senna Tschudin
[...]
Post by Michele Martone
#!/bin/bash
set -e
set -x
trap "read" EXIT # will block for input on error
git clone https://github.com/coccinelle/coccinelle.git
cd coccinelle
./autogen
./configure
make # it stops here
# make install
...
OCAMLOPT enter.ml
OCAMLC main.mli
OCAMLOPT main.ml
OCAMLOPT -o spatch.opt
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:979: undefined reference to `caml_umul_overflow'
/home/abuild/rpmbuild/BUILD/ocaml-4.03.0/otherlibs/bigarray/bigarray_stubs.c:985: undefined reference to `caml_umul_overflow'
collect2: error: ld returned 1 exit status
Error: Error during linking
Makefile:609: recipe for target 'spatch.opt' failed
make: *** [spatch.opt] Error 2
rm parsing_cocci/lexer_cli.ml parsing_cocci/parser_cocci_menhir.ml.d parsing_cocci/parser_cocci_menhir.mli.d parsing_cocci/lexer_script.ml parsing_cocci/lexer_cocci.ml parsing_c/lexer_c.ml
+ read
$ sudo dnf builddep coccinelle
$ sudo apt-get build-dep coccinelle
[...]
Dear Peter, thanks for the apt-get based recipe,
it worked for me on debian buster with ocaml-4.0.5.

Dear Julia, thanks for the pragma patch which will
enable coccinelle to support OpenMP-enabled listings
too (I just tested it on the above setup) :-)

Michele
Michele Martone
2018-05-25 13:56:29 UTC
Permalink
Post by Michele Martone
...
Dear Peter, thanks for the apt-get based recipe,
it worked for me on debian buster with ocaml-4.0.5.
Dear Julia, thanks for the pragma patch which will
enable coccinelle to support OpenMP-enabled listings
too (I just tested it on the above setup) :-)
Michele
Dear Julia,

I have another similar minimal test spatch case failing, attached.
It seems like using
#pragma omp parallel for private(I)
breaks the semantic patch, while having
#pragma omp parallel for
is still ok.
Please see attachment.

I'm using self-built ocaml (e407ecf11 2018-05-25)
and coccinelle (a4532f08 2018-05-17).

would be very useful to have full #pragma resilience!

Michele

Loading...