# How to avoid copying source code for each run

**URL:** https://guildai.org/t/how-to-avoid-copying-source-code-for-each-run/1044
**Category:** General
**Created:** [June 12, 2023, 5:40am UTC](https://guildai.org/t/how-to-avoid-copying-source-code-for-each-run/1044 "2023-06-12T05:40:10Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![kyika](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@kyika](https://guildai.org/u/kyika)
#### Post date: [June 12, 2023, 5:40am UTC](https://guildai.org/t/how-to-avoid-copying-source-code-for-each-run/1044/1 "2023-06-12T05:40:10Z")

</div>

Dear Garrett and other developers,

I would like to know if there is a way to avoid copying the source code for each guild run I executed.

I need this since I am running tens of thousands of guild runs, each executing the same version of a Python module (located in the root directory) but with different command line arguments. As a result, every guild runs have a copy of all the Python files, and eventually, there are so many files under my Guild Home directory (I am on a shared system with a limit on the number of files per user).

I understand the benefit of having a copy of the source code for each run, preventing the code from being changed after the runs are staged, and keeping a definite copy. But for my situation, I would prefer to have independent Guild Homes for different versions of code, and only keep one copy of code for all the guild runs in a Guild Home.

I am wondering if there is a way to do so. One way I can imagine is to package all my Python code and install it to the virtual environment, rather than keep them in my root directory, thus guild has nothing to copy. I would like to know if this is a good idea.

Thanks.

---

<div class="post-metadata">

### Author: ![garrett](https://yyz1.discourse-cdn.com/flex031/user_avatar/guildai.org/garrett/32/224_2.png) [@garrett](https://guildai.org/u/garrett)
#### Post date: [June 12, 2023, 3:02pm UTC](https://guildai.org/t/how-to-avoid-copying-source-code-for-each-run/1044/2 "2023-06-12T15:02:58Z")

</div>

Great questions!

I think your idea of using installed package to meet your Python requirements is a good one. Consider installing using the `-e/--editable` option to `pip install`. This installs a package to the Python env that refers to your project source code. Changes you make to your local project are reflected whenever your package is loaded from Python.

Something like this, from your project dir:

```command
pip install -e .

```

To disable source code copies for an operation, use the `sourcecode` operation attribute like this:

```yaml
train:
  sourcecode: no

```

You can confirm that Guild doesn’t copy any source core using the `--test-sourcecode` option with the `run` command:

```command
guild run train --test-sourcecode

```

You can also check a run using `ls --sourcecode`:

```command
guild ls --sourcecode # should be an empty list

```

All of that said, actual source code should not take up that much space (esp compared to data sets, generated models, images, etc.) You might double-check that you’re copying only what you need for an operation — again use `--test-sourcecode` to see what’s being copied. If there are big files in that list that you don’t need, you can exclude them this way:

```yaml
train:
  sourcecode:
    - exclude: <pattern>
    - exclude: <pattern>

```

---

<div class="post-metadata">

### Author: ![kyika](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@kyika](https://guildai.org/u/kyika)
#### Post date: [June 13, 2023, 9:30pm UTC](https://guildai.org/t/how-to-avoid-copying-source-code-for-each-run/1044/3 "2023-06-13T21:30:02Z")

</div>

Hi Garrett, I have tried your suggestions and it works great! Thank you!
