Flags-dest only delivers defined flags, not config file

Troubleshooting
1

Hi,

I have an issue in which flags-dest only seems to work for flags I define explicitly under flags. I have a Hydra config with nested values and following guildai/examples/hydra at main · guildai/guildai · GitHub .

Using flags-dest: config:<config_path> doesn’t load the values into the Hydra config object when using the decorator, I have to manually load it.

Using flags-dest: globals only delivers global variables that have been defined under flags.

Since manually loading the config file works, with correct nesting and values were overwritten correctly that were specified either via a flag in guild.yaml or as a command line argument, then Guild is loading the config values correctly, right? It just doesn’t deliver overwritten values back to the config file.

Except with flags-dest: globals; then it doesn’t deliver original config values that were left untouched.

My guild.yaml:

train:
      flags-dest: config:embeddings_and_difficulty/configs_hydra/config.yaml
      flags-import: all
      flags:
        label_type: test
      sourcecode:
        - exclude: embeddings_and_difficulty/data
      requires:
        - resource: data
        - config: embeddings_and_difficulty/configs_hydra/config.yaml
          target-path: .guild/sourcecode/embeddings_and_difficulty/configs_hydra
          replace-existing: yes
      main: runner

runner.py:

@hydra.main(version_base=None, config_path='configs_hydra',
            config_name='config.yaml')
def main(args: DictConfig):
   (...)
2

The issue and resolution are discussed here.

You need to change your @hydra.main definition to have config_path='embeddings_and_difficulty/configs_hydra', and note the interface differences in the guild.yml between your train definition and what we’ve put under train-2 here.

Additionally, as a hydra user you may be interested in this RFC: Value dependent flag definitions/flag groups - RFC - Guild AI which should enhance your ability to work with more complex hydra structures.

Let us know if this fixes your issue, or if it still persists.