8000 Search for metadata files in $DATADIR/metadata by evenbrenden · Pull Request #7851 · jgm/pandoc · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Search for metadata files in $DATADIR/metadata #7851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 21, 2022

Conversation

evenbrenden
Copy link
Contributor

If files specified with --metadata-file are not found in the working
directory, look in $DATADIR/metadata.

Closes #5876.

Comment on lines 221 to 227
paths -> mconcat <$>
mapM (\path -> do raw <- readFileStrict path
yamlToMeta readerOpts (Just path) raw) paths
paths -> do
mconcat <$>
mapM
(\path -> do
raw <- readMetadataFile path
yamlToMeta readerOpts (Just path) raw)
paths
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it would be helpful if your change were the minimal one necessary: just changing readFileStrict to readMetadataFile and leaving everything else the same. With the reformatting, it becomes more difficult to see what has changed. (Also, there's now a redundant do.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely, sorry about that. Changed in fixup.

Comment on lines 595 to 597
if existsInWorkingDir then readFileStrict fname
else do
dataDir <- getUserDataDir
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The style of the rest of the pandoc code is:

if foo
   then bar
   else baz

Better to keep to a consistent style.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in fixup.

8000
let path = userDir </> "metadata" </> fname
existsInUserDir <- fileExists path
if existsInUserDir then readFileStrict path
else throwError $ PandocCouldNotFindDataFileError $ T.pack fname
Copy link
Owner
@jgm jgm Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't really the appropriate error since it's not a data file.
Maybe we need a new constructor on PandocError.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could add PandocCouldNotFindMetadataFileError? That would be consistent with readDataFile and readMetadataFile.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added PandocCouldNotFindMetadataFileError.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that a missing metadata file specified with --metadata-dir results in the more general PandocIOError (via readFileStrict) in master. This PR changes that.

@@ -0,0 +1,18 @@
```
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually we just name these with the number, e.g. 5876.md.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to write the other tests that you proposed too. There appears to be a convention for enumerating tests that belong to the same issue, e.g. 5876a.md, 5876b.md. Is that OK?

Should mention that CONTRIBUTING.md suggests using both issue number and feature in the test file names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in fixup.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, didn't realize that about contributing.md!
I guess it's probably not bad advice, and it looks like we've done it sometimes.
So, as you like.

For multiple tests belonging to the same issue, simply include them all in the same file. A command test file can contain any number of command tests (separate code blocks). We don't really have a convention of putting them in separate files with letter suffixes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. I gathered all tests in 5876.md then. I think an extra suffix is unnecessary in this case since the tests are pretty clear about what is being tested.

@@ -0,0 +1,4 @@
---
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need to make sure this test file is included in extra-source-files in pandoc.cabal, or it won't be included in the source distribution.
Maybe also worth testing two further cases:

  • the case where the file exists in the working directory AND in DATADIR/metadata
  • the case where the file exists in neither place (yielding an error)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added resources to extra-source-files as well as two new tests. There's also the case where the file exists in the working directory, but not in the user directory, which is covered by multiple existing tests. I could still add it under this issue for completeness, though.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that you shouldn't put 7851.md and the like in extra-source-files, because it's already covered by a glob (test/command/*.md). However, the yaml file in the subdirectory wasn't covered by a glob. Note that make check-cabal will automatically check to make sure that there aren't committed test files that aren't in the extra-source-files stanza. I don't think we necessarily need the extra test case you describe, but it wouldn't hurt. Up to you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, I ran the check and everything should be included now. I also added a fourth test for when there's no user directory specified and the file does not exist. Then all four possible outcomes of readMetadataFile are covered in 5876.md.

@evenbrenden
Copy link
Contributor Author

Squashed the fixups into a single commit.

6D47 @jgm
Copy link
Owner
jgm commented Jan 21, 2022

This looks great! Just a couple small things now.

  1. Update list of Exit Codes in manual (heading # Exit codes) to add the new 98 code
  2. Modify your commit message so it clearly marks the two API changes with [API change]. These are the new constructor on PandocError and the new exported function from T.P.Class (via PandocMonad).

If files specified with --metadata-file are not found in the working
directory, look in $DATADIR/metadata.

Expose new readMetadataFile function from T.P.Class (PandocMonad)
[API change].

Expose new PandocCouldNotFindMetadataFileError constructor from
T.P.Error (PandocError) [API change].

Closes jgm#5876.
@evenbrenden
Copy link
Contributor Author

This looks great! Just a couple small things now.

1. Update list of Exit Codes in manual (heading `# Exit codes`) to add the new 98 code

2. Modify your commit message so it clearly marks the two API changes with `[API change]`.  These are the new constructor on PandocError and the new exported function from T.P.Class (via PandocMonad).

Like this?

Copy link
Owner
@jgm jgm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great.

@jgm jgm merged commit 7df29e4 into jgm:master Jan 21, 2022
@jgm
Copy link
Owner
jgm commented Jan 21, 2022

Now that I've merged this, a potential issue occurs to me.
(Perhaps it also arises for data files and templates.)
Suppose someone says --metadata-file ../foo.yaml. If foo.yaml isn't found in the parent directory of the working directory, do we really want to look for it in the parent of the user data directory, which would be outside of the user data directory?

@evenbrenden evenbrenden deleted the metadata-directory branch January 21, 2022 22:08
@evenbrenden
Copy link
Contributor Author

Hm, can't imagine that anyone wants to do that on purpose. I checked the behaviour for --template, which is different: If the path specified isn't found directly, $DATADIR/templates is searched by filename. That is, the path is stripped. This means that you can't go outside of the data directory, but also that you can't search subdirectories of $DATADIR/templates. Not sure what the best solution is, but I think it should be consistent for any file type. I can open an issue for further discussion.

@jgm
Copy link
Owner
jgm commented Jan 22, 2022

One approach would be to allow subdirectories in both cases, but only if they don't contain ...
But yeah, we should track this on a separate issue.

@evenbrenden
Copy link
Contributor Author

Created #7861.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support metadata directory in data directory
2 participants
0