Bug 24239

Summary: .gitignore enhancement: common template with default values for components
Product: xorg Reporter: Gaetan Nadon <memsize>
Component: Build/ModularAssignee: Gaetan Nadon <memsize>
Status: RESOLVED FIXED QA Contact: Xorg Project Team <xorg-team>
Severity: enhancement    
Priority: low CC: alan.coopersmith, peter.hutterer
Version: gitKeywords: janitor
Hardware: x86-64 (AMD64)   
OS: Linux (All)   
Whiteboard:
i915 platform: i915 features:
Bug Depends on:    
Bug Blocks: 23814, 24206    
Attachments:
Description Flags
Comprehensive example using xserver .gitignore files
none
The .gitignore template with default values for compoennts none

Description Gaetan Nadon 2009-09-30 17:46:35 UTC
I noticed the .gitignore files require frequent maintenance and are hard to read. The main cause is the long list of generated files that developers shouldn't really have to get involved with. I scanned, merged and filtered all.gitignore files and realized there was a common set of exclusion patterns. I cross-referenced them with GNU tools docs to validate and complete the list.

The concept of the replacement .gitignore is a template with a default section and a custom section.  The default section is all of the obscure generated stuff about Autotools, compiler, linker, etc...  This section is not to be edited by component developers. The custom section is where one can add files to ignore that are component specific and known to the developers. One can also override the default section in rare cases where a name clashing occur, a file type is misused or a generated file is checked-in.

The template also applies by default to all subdirectories. Additional .gitignore files can be created in subdirectories for the same add and override operations. This leaves the top level file cleaner, not having to list stuff for the subdirectories which is commonly done now. Out of 268 components, 145 are happy with just the template and no customization. The rest have a 1 to 3 lines of custom patterns. The xserver top level file has 4 lines in the custom section and only 27 .gitignore files in subdirectories with an average of 3 lines.

What's the risk of missing an exclusion pattern? Mainly in the custom section of components which have subdirectories that do not build from build.sh (testing or obsolete stuff) or do not build on my platform (SUN, MAC). I looked for them manually, but could have missed some. 

What's the risk of erroneously ignoring a file? Next to none. Using 'find' with the exclusion patterns on an unbuilt tree will reveal all the files that would be erroneously ignored.

Attached are the template itself and a tar file with all the .gitignore files for the xserver which provide a comprehensive example.
Comment 1 Gaetan Nadon 2009-09-30 17:48:04 UTC
Created attachment 29965 [details]
Comprehensive example using xserver .gitignore files
Comment 2 Gaetan Nadon 2009-09-30 17:48:53 UTC
Created attachment 29966 [details]
The .gitignore template with default values for compoennts
Comment 3 Peter Hutterer 2009-10-04 16:37:46 UTC
for some reason, bugzilla doesn't give me the attachments right now, so I have to write part of this from memory, I looked at the template a few days ago
I like the idea in principle, though a few questions I have here. 

- Do we want to continue using .gitignore in subdirectories? I'm not a big fan of having those sprinkled in "random" directories though I acknowledge that this is current practice. Though not without problems: for example, the xserver gitignore has overlapping .gitignore rules from the top-level and the subdirectory .gitignores. IMO in the long-term it'll be cleaner to have a single .gitignore.

- I think the template needs massive warnings in the top of the file to make sure nobody tries to add to the common section without updating all the other templates too. Is there anything we can do to avoid becoming inconsistent?
 
> What's the risk of erroneously ignoring a file? Next to none. 

IMO, this is the biggest risk since one may forget to git add a new file and you won't notice until other people are shouting at you. Nothing easily fixable but it leaves the server unbuildable until that file is added again. Such commits should be avoided at all costs. 
Comment 4 Gaetan Nadon 2009-10-04 18:07:24 UTC
(In reply to comment #3)
> for some reason, bugzilla doesn't give me the attachments right now, so I have
> to write part of this from memory, I looked at the template a few days ago
> I like the idea in principle, though a few questions I have here. 
> 
> - Do we want to continue using .gitignore in subdirectories? I'm not a big fan
> of having those sprinkled in "random" directories though I acknowledge that
> this is current practice. Though not without problems: for example, the xserver

I have thought about it a lot. Centralize vs decentralize. I noted many entries were made as paths rather than pattern. So test/video/regdump wasn't meant as a pattern but as file in a particular location. There can't be more than one test/video directory. With this mind set, it is natural to think of .gitignore as a high level list of paths.

But this isn't what .gitignore is about, the way I understand it. You specify file stem patterns which apply in whatever directory they happen to be. For example *.o or Makefile. You don't want to list hw/dmx/Makefile. 

Having said this, it also depends who is updating .gitignore. If it is one person who knows it all, it might be preferable to centralize. On the other hand, if subdirectories come and go, it's better to be decentralized. when a directory is removed, so is the .gitignore. I saw a lot of pollution in the toplevel one.

It won't be possible to to never have a subdir .gitignore, for example there is a config.h in a subdir which is a normal header file. Unfortunatly, I had to write a few subdir .gitignore because of the man page generated from Docbook. Some generated files are checked-in which are ignore by the toplevel.

Using the template does not enforce one method over the other. I noticed there were already files in subdirs so I followed this pattern.

> gitignore has overlapping .gitignore rules from the top-level and the
> subdirectory .gitignores. IMO in the long-term it'll be cleaner to have a
> single .gitignore.
> 
> - I think the template needs massive warnings in the top of the file to make
> sure nobody tries to add to the common section without updating all the other
> templates too. Is there anything we can do to avoid becoming inconsistent?
> 

If the default section is changed, there won't be any harm done in the component so as long as he knows what he is doing. But we lose in consistency. A post-build tool checker is the only thing I can think of. Large projects have this for policy enforcement.

> > What's the risk of erroneously ignoring a file? Next to none. 
> 
> IMO, this is the biggest risk since one may forget to git add a new file and
> you won't notice until other people are shouting at you. Nothing easily fixable
> but it leaves the server unbuildable until that file is added again. Such
> commits should be avoided at all costs. 
> 

That statement was referring to the techniques I used to create the template. I did not mean that using the template will prevent erroneously ignoring a file. Apologies.

Using a template will not contribute to this problem unless it is poorly designed. For instance, if I had included lex.c because I saw one place using it, it could ignore a valid (none generated) lex.c file somewhere. 

That's why the default section only contains known generated files that were already present and were cross-referenced by the documentation and searched over the Internet for use cases.

If we are to generate the INSTALL file then I need to add it to the template. If there is a non-generated INSTALL file in a sub dir, you can add a .gitignore with !INSTALL or add in the toplevel one !subdir/INSTALL. 

Thanks, this was a good workout.






Comment 5 Gaetan Nadon 2009-11-10 07:36:21 UTC
Fix typo in title
Comment 6 Gaetan Nadon 2009-12-01 18:14:44 UTC
All patches applied.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.