Bug 554280

We've already had a small discussion about how rlFileBackup should
act upon missing files:

    https://bugzilla.redhat.com/show_bug.cgi?id=478981#c1

The output was that we log the missing files but the command
should report a non zero exit status as for backups we need to be
sure that they really went well.

Anyway, in your case I think you can simply do:

    rlFileBackup --clear "/etc/gcrypt/"

Adding the fips_enabled file for testing purposes will then be ok
as it will be cleared once the rlFileRestore is called. Will that
work for you?

Well, it will work. But it seems to me insane to backup whole directory (let's suppose there may be several files) just because of the one file inside the directory that should be deleted (what if the backup fails for a different file inside that directory?). To be clear, I was not looking for a workaround (which would work really great in this case) but for a comfortable way how to ensure such a behaviour. 

> The output was that we log the missing files but the command
> should report a non zero exit status as for backups we need to be
> sure that they really went well.

Therefore I have mentioned that this behaviour should not be used by default. But if I am sure with the filename, I can't see any problem here. 

Never mind, Now I will use the command you have told me. Maybe one day there will be demand for such a feature. :-)

Oh, I see. Then it's only a question of error logging:

    rlFileBackup --clean /etc/gcrypt/fips_enabled
    or
    rlRun "rlFileBackup --clean /etc/gcrypt/fips_enabled" 0,1

will do exactly what you need, except from the error message
displayed. Seems to me rather a cosmetic RFE, but we could add
something like --force or --missingok option to suppress the
message.

Or: There are different exit codes returned by rlFileBackup, we
could perhaps reserve a special value for "does not exist" which
then could be used in the rlRun's expected values.

Yep, it would be great to be able to avoid those error messages. Let's see if anyone else is also interested in such and update.

> Seems to me rather a cosmetic RFE, but we could add
> something like --force or --missingok option to suppress the
> message.

+1 to --missingok :-)

Again I'm facing the problem of backing up possibly unexisting file. BTW it is /etc/named.conf which is present in the rhel6 system by default, but in rhel5 system it is not. 

Basically I have to do something like 

[ -f /etc/named.conf ] && rlFileBackup /etc/named.conf || CLEANUP="$CLEANUP /etc/named.conf"

...

rlFileRestore
rm -rf "$CLEANUP"

this is crazy!

Additionally if a single file (whose existence can not be guaranteed) must be backed up, we need to initialize the backup mechanism with something like

rlFileBackup "`mktemp -f`"

which is also crazy.

Another possibility, which was mentioned somewhere yet, is to back up whole the parent directory. But this can lead to unrecoverable system fault for example while backing up whole the /etc, then modifying mtab and running rlFileRestore in the end.

I think --missingok switch should be added for sure at least.

Or are there some other ideas? Please, tell. I may have missed something..

Seems there's quite consensus about this feature. I'm going to
implement the new --missingok option. Or --missing-ok to get
around the word mess? Does anybody have a better name for that?

Until the new feature hits production repos you might want to use
the backup functions outside the assert or with proper expected
status as mentioned in comment #3.

    rlFileBackup --clean /etc/named.conf
    rlRun "rlFileBackup --clean /etc/named.conf" 0,8

Thank you Petr. I was thinking about other option (--keepclean or --mrproper :-)) which would replace '--clean --missing-ok' since you would have to use both options anyway to get requested functionality. 

I mean, suppose you would use --missing-ok only. During the test the file may be created and it won't be deleted later on. This may cause problems. Therefore I would prefer new option with '--clean --missing-ok' functionality insted of single '--missing-ok' option.

This is really important. Please make any conclusion about the switch name and implement it somehow. Setting Urgent priority and severity.

(In reply to comment #8)
> Thank you Petr. I was thinking about other option (--keepclean or --mrproper
> :-)) which would replace '--clean --missing-ok' since you would have to use
> both options anyway to get requested functionality. 
> 
and IMO that's how cmdline options should be used. Think of it as an instance of the "do one thing and do it well" principle.  

> I mean, suppose you would use --missing-ok only. During the test the file may
> be created and it won't be deleted later on. This may cause problems. 

Suppose you'd like to use --missing-ok but you won't have it ...
More powerful tools can lead to bigger problems, true.

>Therefore
> I would prefer new option with '--clean --missing-ok' functionality insted of
> single '--missing-ok' option.

Two options (usable both separately and together) IMHO are superior from both functionality perspective (I can do more with them) and usability (clear  functionality, without overlaps is IMHO easier to grasp and remember)

-1 to aggregating  '--clean' and '--missing-ok' in one new option.

Agree. -1 to aggregating it as it may lead to regressions. I will gladly write a bit longer command if it will work all right :-) 

What do you think about this stuff, Petr?

I can understand that two separate options are "more powerful" than their conjunction. But I think that people will tend to use --missing-ok badly (without --clean). Anyway, I will be (more or less) happy with any solution. :-)

I misunderstood it in my Comment #11: No regression menaces here as --clean will stay untouched anyway. But now I'm not sure if it makes sense to have --missing-ok without the --clean  functionality. Maybe we should rather aggregate.

But still: I will gladly write a bit longer command if it will work all right :-)

Ok, let's sum up possible scenarios to make it clear:

rlFileBackup --clean file

    file exists ---> backed up & cleaned & restored
    file missing ---> fails with an error

rlFileBackup --clean --missing-ok file 

    file exists ---> backed up & cleaned & restored
    file missing ---> no backup, but cleaned (if exists) during restore

rlFileBackup --missing-ok file 

    file exists ---> backed up & restored (no cleanup)
    file missing ---> no backup, no cleanup, no restore

From my point of view, all three scenarios are possible and
usable and useful. Is that what we want here? If agreed, I'll
implement the feature, hopefully by the next week.

I'm quite fine with this solution. Karel, Ales, anyone, what do you think?

OK. If you think that "no backup, no cleanup, no restore" is useful.. :-)

Anyway, be aware of the situation when the only file to backup is the one which is missing (so rlFileRestore is not complaining)

Thank you very much Petr.

*** Bug 692392 has been marked as a duplicate of this bug. ***

Well, now crucial questions are:

* What is the status here?

* What is the ETA for this?

* What is (and should be [regardless of regressions]) the default behaviour? [*]

[*] Since, from (my) QA point of view, a system state is an invariant for (backup & test & restore) sequence, i.e. "rlFileBackup file" should work as "rlFileBackup --clean --missing-ok file" (see Comment #14).

(In reply to comment #18)
> Well, now crucial questions are:
(...)

> [*] Since, from (my) QA point of view, a system state is an invariant for
> (backup & test & restore) sequence, i.e. "rlFileBackup file" should work as
> "rlFileBackup --clean --missing-ok file" (see Comment #14).

I thought about this a little, and yes, I think this is the most common use case. The question is how to implement this. 

1) If we make this default, we would need to provide new options for *disabling* the --clean and --missing-ok functionality, and we would need to deprecate (as in: still present, but no effect) already existing --clean option. 

2) Or we can provide new rlFileBackupSmart wrapper, using rlFileBackup with proper options.

I'm more inclined towards 1), but not against 2) either.

I wouldn't make "--clean --missing-ok" the default. In a test
rlFileBackup file also work as rlAssertExists file (it complaints it the file doesn't exist) - I do not want to explicitly write rlAssertExists prior the every backup.

I would prefer just to add --missing-ok option and even better also --mrproper (--whatever) option as an alias to "--clean --missing-ok".

Just a short summary of current functionality (after 42 months):

Currently "rlFileBackup --clean nosuchfile" adds nosuchfile on a clean list so it is deleted by the rlFileRestore command in case it was created during the test run. This is good.

On the other hand the test produces two errors:

  :: [  ERROR   ] :: rlFileBackup: File /tmp/x does not exist.
because the file didn't exist

and 
  cp: cannot stat `/var/tmp/beakerlib-PDosjRd/backup/*': No such file or directory
:: [ 14:01:15 ] :: [ ERROR   ] :: rlFileRestore: Failed to restore files from /var/tmp/beakerlib-PDosjRd/backup
becase nosuchfile was the only file to backup.

I am still forced to use workarounds to avoid these errors such as 
 [ -f nosuchfile ] || rlFileBackup nosuchfile 
 rlFileBackup /etc/fstab                 # just to silent rlFileRestore
..
 rm -f nosuchfile
 rlFileRestore

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

I am definitely for something like --missing-ok. I think it makes sense to use it with --clean so may be we could make it default in --clean directly.

I also use --clean as default. Personally I would not be afraid to make --clean --missing-ok as default as long as the cleanup processing is smart enough not to remove everything first and restore what should be restored. But it is another story, see bz980153.

fixed by https://git.fedorahosted.org/cgit/beakerlib.git/commit/?id=62610686d23f185a3174fee2b367c5491aaa54cf

added missing part in getopt by https://git.fedorahosted.org/cgit/beakerlib.git/commit/?id=1462a71190de414eb578f04929168bc63439c5e0