Bug 1283843

Summary:	newer ignored
Product:	[Fedora] Fedora	Reporter:	Richard Jasmin <spike85051>
Component:	tar	Assignee:	Pavel Raiskup <praiskup>
Status:	CLOSED NOTABUG	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	23	CC:	kdudka, ovasik, praiskup, spike85051
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-12-21 07:34:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Richard Jasmin 2015-11-20 05:02:00 UTC

Description of problem:
Using the following to make an incremental only partially works:

#sometime in the past..yields: date-month
#NOW=`date +%d-%b`
#echo $NOW > full-date

NEWER="--newer `cat full-date`"

tar $NEWER -cP --exclude-backups --exclude-caches-all --exclude-vcs --exclude=.cache -f backup-inc.tar /home/user/

using find and pipe to commands like cp,mv,rsync has worse results. For some reason linux likes to fudge dates and access times no matter what. There simply isnt a workable solution to fetching files made or modified after date X. (And doing it repeatedly to create a logarithmicly shrinking backup set)

I am aware of options like TimeMachine on Linux but these operations have one flaw:

CPU is constantly pegged while in use, diff-ing everything as you go about your business.It would be better to do this once a week or once a month or so.Apparently nobody told Jobs about this.The effect was mirrored on linux.

In addition I am spliting the result for hard copy backup onto bluray. I plan on full sets every six months or so.

So whats the fix?

What winds up happening for example is a massive container keeps getting thrown into the incremental set on an ongoing basis(along with other files not modified) , even if I touch the file with a date in the past(which displays correctly). Newer files are there but a lot of older files are too. When you check the older files, they are indeed modified in the past(beyond the requested backup date), but are now accessed when backing them up.Yet somehow they are included in the newer backup set.

This is my second incremental set. Its as if the 'newer' date is being ignored.

For further example:
1-Oct (full) no date
13-Oct (in the file-newer date)
and 1-Nov (lets say now, today)

Some files from 1-Oct are getting put here.

Version-Release number of selected component (if applicable):
23, maybe 22(forced to downgrade here in a minute)

How reproducible:
every run

Steps to Reproduce:
see above

Actual results:
something is causing the modification date to be mis-read

Expected results:
only files beyond date X in past should be included in set. I am using actual date but if using find for example, you can set -/+30.

Comment 1 Pavel Raiskup 2015-11-20 06:56:12 UTC

Richard, thanks for this report but I got lost.

The --newer option compares against ctime, --newer-mtime against mtime.  Did
you want to use --newer-mtime instead?

Comment 2 Richard Jasmin 2015-12-17 19:25:12 UTC

not sure.
I dont want "today".
+/-30 would suffice but how to do this above you need an actual date like 1-Nov. I want some date in the past.IE: my last backup date.

"newer" expects one day past that(for repeated instances) but is having an issue checking the file date for some reason.

mtime sounds correct. Again a documentation glitch. Sound be documented but may not be.

Now we are getting windows dates in here too. IE: if a file was modified on windows vice linux. Do I need to touch everything and try a full backup again? Because if I touch everything and do an incremental, I might as well be doing a full.

But shouldnt other simple commands like: move, copy,... 

use this as well? No matter what "time" option Ive used, Im geting the same result, as if Linux cant track dates for some reason.I dont believe Ive disabled access time on /home.

What you said seems to do the trick but only with tar.

Comment 3 Pavel Raiskup 2015-12-21 07:34:40 UTC

(In reply to Richard Jasmin from comment #2)
> not sure.
> I dont want "today".
> +/-30 would suffice but how to do this above you need an actual date like
> 1-Nov. I want some date in the past.IE: my last backup date.
> 
> "newer" expects one day past that(for repeated instances) but is having an
> issue checking the file date for some reason.
> 
> mtime sounds correct. Again a documentation glitch. Sound be documented but
> may not be.

I think it is documented, try:

  $ info tar "after"

> Now we are getting windows dates in here too. IE: if a file was modified on
> windows vice linux. Do I need to touch everything and try a full backup
> again? Because if I touch everything and do an incremental, I might as well
> be doing a full.

I don't understand this part of your response.  As far as 'mtime' comparison
works for you, don't hesitate to use it.

> But shouldnt other simple commands like: move, copy,... 
>
> use this as well? No matter what "time" option Ive used, Im geting the same
> result, as if Linux cant track dates for some reason.I dont believe Ive
> disabled access time on /home.

The 'mv' and 'cp' from coreutils have '-u' option which also looks at 'mtime'.
But I'm not the right person to give you the help here, I would suggest you
to contact upstream coreutils.

> What you said seems to do the trick but only with tar.

Maybe I have difficulties to understand the issue - but it looks the tar is
fine here.  Richard, please reopen if you feel otherwise.