8000 Problem with adaptation of vep apptainer version · Issue #379 · imgag/megSAP · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Problem with adaptation of vep apptainer version #379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
alex-seitz opened this issue Apr 11, 2025 · 5 comments
Open

Problem with adaptation of vep apptainer version #379

alex-seitz opened this issue Apr 11, 2025 · 5 comments

Comments

@alex-seitz
Copy link
Contributor

Hi,
we typically add the "SpliceRegion" vep plugin to our megSAP analysis, because our biologists need the expanded splicing annotations.
Additionally, we adapted the SpliceRegion.pm-file to include all variants from -20/+3 to -3/+8 around the splice sites. However, I can't get the installation of the plugin to work.
When I adapt the db_download.php line like this:
$parser->execApptainer("vep", "INSTALL.pl", "--SPECIES homo_sapiens --ASSEMBLY GRCh38 --AUTO acp --NO_UPDATE --NO_BIOPERL --CACHEDIR $vep_data_dir/cache --CACHEURL $vep_data_dir/ftp --NO_TEST --NO_HTSLIB --PLUGINS SpliceRegion", [$vep_data_dir], [$vep_data_dir]);
I get the error
ERROR: Could not make directory ./Bio
When I try to run in manually like this:
apptainer exec -B <megSAPdir>/data/dbs/ensembl-vep-112/ <megSAPdir>/data/tools/apptainer_container/vep_release-112.0.sif INSTALL.pl --SPECIES homo_sapiens --ASSEMBLY GRCh38 --AUTO acp --NO_UPDATE --NO_BIOPERL --CACHEDIR <megSAPdir>/data/dbs/ensembl-vep-112/cache --CACHEURL <megSAPdir>/data/dbs/ensembl-vep-112/ftp/ --NO_TEST --NO_HTSLIB --PLUGINS SpliceRegion
I get the error
No such file or directory at /opt/vep/src/ensembl-vep/INSTALL.pl line 778.
Do you have a solution for me or maybe another way to get an extended SpliceRegion annotation?

Best,
Alex
@KilianIlius
Copy link
Collaborator

Hey Alex,

regarding your db_download.php adaptation you have to include your cwd to the bind paths. Our execApptainer() function adds the --no-mount home,cwd and --cleanenv flag to each container execution to make sure no unwanted files are mounted into the container, which could conflict with container execution.

It seems that your plugin installation tries to create a folder in your cwd which is not writable for apptainer since cwd is not mounted.

Since the vep container is one of the only containers I didn't build but pulled from the official ensemble-vep page from dockerhub (https://hub.docker.com/r/ensemblorg/ensembl-vep) I'm not sure why it can't find the INSTALL.pl script in your manual execution.

Can you check if the script in the container can be found with your specified bind paths?

apptainer exec -B <megSAPdir>/data/dbs/ensembl-vep-112/ <megSAPdir>/data/tools/apptainer_container/vep_release-112.0.sif ls /opt/vep/src/ensembl-vep/

If you can find the INSTALL.pl script in the expected location you can try to execute the script with it's full path:

apptainer exec -B <megSAPdir>/data/dbs/ensembl-vep-112/ <megSAPdir>/data/tools/apptainer_container/vep_release-112.0.sif /opt/vep/src/ensembl-vep/INSTALL.pl --SPECIES homo_sapiesn ...

Hope that helps, let me know if it doesn't ;)

Cheers,
Kilian

@alex-seitz
Copy link
Contributor Author

Hi,
thank you for the Tips.
The INSTALL.pl is present in the given ls command, however, when I use the full path to the
/opt/vep/src/ensembl-vep/INSTALL.pl
I still get the same error:
No such file or directory at /opt/vep/src/ensembl-vep/INSTALL.pl line 778
When I remove the cwd from the
$apptainer_args[] = "--no-mount home,cwd";
I get the same error, when running the db_download.php script
ERROR: 'Call of external tool 'INSTALL.pl' returned error code '2'.' in /shared_software/megSAP_GRCh38_20250326/src/Common/ToolBase.php:941.
So It does not with or without the cwd.
Strange that it really is present within the container but can't be found when called without the cwd

@KilianIlius
Copy link
Collaborator

I'll try to replicate the problem and keep you updated on any solution ideas.

@KilianIlius
Copy link
Collaborator

Hey Alex,

I found the problem. When trying to install an ensemble-vep plugin using the container INSTALL.pl tries to download files to /opt/vep/src/ensembl-vep inside the container. Since the container is read-only it cannot download the file which seems to fail silently.
In INSTALL.pl line 778 this file is needed and not found, since it wasn't downloaded.

You can circumvent this issue by copying the folder /opt/vep/src/ensembl-vep from inside the container to your host system and bind mount it during the installation execution:

apptainer exec data/tools/apptainer_container/vep_release-112.0.sif cp -r /opt/vep/src/ensembl-vep/ .

apptainer exec --no-mount home,cwd --cleanenv 
-B ensembl-vep/:/opt/vep/src/ensembl-vep/,
<megSAP_data>/dbs/ensembl-vep-112,
<megSAP_data>/dbs:
vep_release-112.0.sif INSTALL.pl 
    --SPECIES homo_sapiens 
    --ASSEMBLY GRCh38 
    --AUTO acp 
    --NO_UPDATE 
    --NO_BIOPERL 
    --CACHEDIR <megSAP_data>/dbs/ensembl-vep-112//cache 
    --CACHEURL <megSAP_data>/dbs/ensembl-vep-112//ftp 
    --NO_TEST 
    --NO_HTSLIB
    --PLUGINS SpliceRegion

Output:

Setting up directories

Downloading required Ensembl API files
 - fetching ensembl
 - unpacking ./Bio/tmp/ensembl.zip
 - moving files
 - getting version information
 - fetching ensembl-variation
 - unpacking ./Bio/tmp/ensembl-variation.zip
 - moving files
 - getting version information
 - fetching ensembl-funcgen
 - unpacking ./Bio/tmp/ensembl-funcgen.zip
 - moving files
 - getting version information
 - fetching ensembl-io
 - unpacking ./Bio/tmp/ensembl-io.zip
 - moving files
 - getting version information
 - getting list of available cache files
 - copying <megSAP_data>/dbs/ensembl-vep-112//ftp/homo_sapiens_vep_112_GRCh38.tar.gz
 - unpacking homo_sapiens_vep_112_GRCh38.tar.gz
 - converting cache, this may take some time but will allow VEP to look up variants and frequency data much faster
 - use CTRL-C to cancel if you do not wish to convert this cache now (you may run convert_cache.pl later)
2025-04-11 17:39:26 - Processing homo_sapiens
2025-04-11 17:39:26 - Processing version 112_GRCh38
2025-04-11 17:39:26 - No unprocessed types remaining, skipping
2025-04-11 17:39:26 - All done!

 - installing "SpliceRegion"
SpliceRegion already installed; overwriting
 - add "--plugin SpliceRegion" to your VEP command to use this plugin
 - OK

All done

But it seems like the SpliceRegion plugin is already installed inside the container. To utilize it you probably need to add --plugin SpliceRegion to the vep command as described in the output. We don't have an easy option to customize the arguments passed to vep, but you can add $args[] = "--plugin SpliceRegion" to the script src/Tools/an_vep.php line 78.

@alex-seitz
Copy link
Contributor Author

Thank you, I will try that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alex-seitz @KilianIlius
0