[gentoo-soc] ventoo application - gentoo-soc - Gentoo Mailing List Archives

From:	Christopher Harvey <chris@××××××××××××.com>
To:	gentoo-soc@l.g.o
Subject:	[gentoo-soc] ventoo application
Date:	Mon, 29 Mar 2010 18:28:04
Message-Id:	`4BB0F139.8010703@basementcode.com`

1

Hi list,

2

here is a working copy of a gosc '10 application. I know it's long, but 

3

there are some open questions at the end I'd like people to at least 

4

take a look at if you're going to read.

5

6

Thanks.

7

8

-------------------------

9

10

Google summer of code 2010 Application

11

12

Ventoo

13

14

15

Ventoo is a visual configuration editor that lets the user graphically 

16

edit system configuration files. Ventoo was originally written by Luis 

17

F. Araujo in '07, but was never finished, as it is a large project. All 

18

the existing code is in python and GTK. This summer I'd like to work on 

19

Ventoo until it is useable and can be put into portage. I have several 

20

ideas that I will outline below that I think would make Ventoo useable 

21

by the average gentoo user. They are listed in order of importance. The 

22

following is just a list, there are implementation details after this list.

23

24

25

Use http://augeas.net/ to parse and write configuration files. By using augeas I don't need to maintain python scripts to parse each configuration file. I can just use a grammar description. This is far less error prone and much more maintainable than ad-hoc parsing. Augeaus turns configuration files into abstract syntax trees and back. Augeaus is already in portage. Parsing languages is hard without a grammar, so using an AST is the only way I would trust a program to modify a configuration file on any system. Augeaus is written in C, but has python bindings.

26

27

Support for remote editing on multiple computers simultaneously. It would be really nice if ventoo could be installed on one computer, then connect to several computers, parse their configuration files, then apply changes across all of them. If this is used along with ASTs then the configuration files wouldn't even have to be the same on each computer. I ask for help here, I don't know the best way to implement this. Maybe just scp to make the changes? I'm assuming that all the computers have the same root password and it would be used in an administration setting with many many computers. What's the best way to handle multiple computers with different configurations? Can we make this really slick and powerful? See implementation details below.

28

29

Parse the comments that generally appear above each configuration variable and display them to the user as documentation. Most Gentoo configuration files have comments that appear just above the option to configure, these can also be parsed into an AST. In the implementation section there are details on providing custom documentation.

30

31

Use the python to provide smart completion within the visual config editor. Eg: In /etc/conf.d/consolefont CONSOLEFONT is set to something listed in /usr/share/consolefonts Would be nice to have scripts what list and return possible options for variables like this. It could be as simple as hardcoded values, or dynamically listed values from python/bash/whatever scripts for each system and configuration file. Hopefully some of the existing code can be adapted to help with this.

32

33

Display config file changes as a diff file. Optionally highlighted with a 3^rd    party program like meld, or library like gtksourceview. I say optionally because I think it's best to keep dependencies for a program like this to a bare minimum.

34

35

Integrate rc-update and a few eselect modules. Eg, select a kernel from a list.

36

37

Optionally provide hints in the ventoo modules about variable types. For example a path variable would prompt the user in an open dialogue. An enumeration with a dropdown box. Default to a plain text box.

38

39

*Implementation Details*

40

41

I am proposing to have separate modules manage the grammars for the configuration files. I am imagining that these modules would be stored in /usr/share/ventoo. Each module is defined as a grammar, code completion scripts, and variable/value documentation. The only required component of a module is the grammar so that Augeaus can parse the file and write it back. Code completion scripts return possible values to be entered into specific variables in configuration files. Example, if a variable asks for a specific device hard disk on the system then a code completion module would return ls /dev/sd* /dev/hd*.

42

Right now ventoo uses an html browser to display ventoo specific help. I propose we use this existing dependency for displaying the optional documentation mentioned above. If the parsed comments from a configuration file are not sufficient then the ventoo module maintainer could write additional documentation as an html file that ventoo could display as the user is editing that variable.

43

The fact that modules are completely separate from the core ventoo code means that a packages that have configuration files can provide ventoo with these module files and and visual configuration file editing support for themselves. Kind of like emacs mode use flags, packages add their own emacs modes.

44

45

If ventoo is editing several computers at the same time then things could get complicated when configuration files aren't the same for all of them. I think realistically it would be impossible to implement this properly over one summer, but room should definitely be made in the gui and in the code for future work here. This summer I imagine only implementing remote editing for a single computer at a time over scp and ssh and all the while adding comments about how to start off with multiple computers. The only hard part here is how to deal with different computers that have different configuration files and only some options need to be edited. How do we display this information intelligently to the user?

46

47

*Deliverables*

48

49

This is a list things that I promise to implement over the summer to pass the GsoC project.

50

51

*

52

53

      Augeas grammars for most configuration files on a standard unix system.

54

55

*

56

57

      Displaying important nodes in an AST

58

59

*

60

61

      Writing ASTs back to files

62

63

*

64

65

      Displaying diff generated from ventoo

66

67

*

68

69

      Remote configuration of one other computer at a time

70

71

*

72

73

      rc-update integration

74

75

*

76

77

      Working documentation displayed in embedded html browser

78

79

*

80

81

      Optional custom documentation

82

83

*Timeline:*

84

85

May 26^st    to June 2^nd  : Write Augeas grammars, start writing code to walk through ASTs

86

June 2^nd    to June 9^th  : Have a basic gui to display config file variables.

87

June 9^th    to June 16^th  : Make the GUI nicer, integrate rc-update, diff files

88

June 16^th    to June 23^rd  : Buffer zone, make sure everything is perfectly stable up to this point. If things take longer than expected use this week. If things go smoothly use this week to implement extras, like nice documentation modules and nice code completion modules. Make sure that the program can parse some standard config files, display the diff and write back. Complete midterm evaluations. At this point the application would be able to read/write ASTs and have a few ventoo modules.

89

90

By July 14^th    make code completion scripts working. Make GUI handle complex config scripts, example, fstab. Fstab has rows and columns, each with different types therefore the GUI can't just be a linear list of variables, it has to be able to display a matrix with different types in each cell.

91

92

At this point there are 4 weeks left to fix bugs, write extensive and complete ventoo modules and ventoo documentation. Create ebuilds. Test on multiple systems. Edit other packages like cups or samba to have their own ventoo modules that could be enabled with a ventoo use flag. This region of time can also act as a buffer to make sure that there is enough time to complete everything on the above list so that ventoo useable in practice by an end user. If everything goes perfectly there may even be time to work on editing multiple computers at the same time.

93

94

There are a few points I especially want to get some input on.

95

Implementation language? This project is written python. I like python, but I like C and C++ more. I like to use glade with gtk-buidler but the current code has widgets hardcoded in.

96

97

Separate GUI?

98

Can we separate the GUI code from the internal workings of ventoo? Maybe others would like to add their own GUIs.

99

100

How important is it to keep old code?

101

I'm asking this because right now ventoo uses python to parse config files. It also has the GUI coded in with the project as the parsing. IMO these are both weaknesses but at the same time it kills me to rewrite what has already been done. I'd rather move forward keeping existing efforts.

102

103

I also want to point out that writing a GUI should never be a big deal...after all it's basically just putting widgets in a window. So long as the GUI is modular and isn't tied in too tightly with the inner workings of a program I don't mind doing it in python, glade or otherwise. I feel like this isn't my call so I'm asking the people who make these decisions to let me know what has the best chance of getting accepted.

104

105

Thanks.

106

- Christopher Harvey

107

Computer Science, 3^rd    year, McGill University.

Gentoo Archives: gentoo-soc

Replies

1	Hi list,
2	here is a working copy of a gosc '10 application. I know it's long, but
3	there are some open questions at the end I'd like people to at least
4	take a look at if you're going to read.
5
6	Thanks.
7
8	-------------------------
9
10	Google summer of code 2010 Application
11
12	Ventoo
13
14
15	Ventoo is a visual configuration editor that lets the user graphically
16	edit system configuration files. Ventoo was originally written by Luis
17	F. Araujo in '07, but was never finished, as it is a large project. All
18	the existing code is in python and GTK. This summer I'd like to work on
19	Ventoo until it is useable and can be put into portage. I have several
20	ideas that I will outline below that I think would make Ventoo useable
21	by the average gentoo user. They are listed in order of importance. The
22	following is just a list, there are implementation details after this list.
23
24
25	Use http://augeas.net/ to parse and write configuration files. By using augeas I don't need to maintain python scripts to parse each configuration file. I can just use a grammar description. This is far less error prone and much more maintainable than ad-hoc parsing. Augeaus turns configuration files into abstract syntax trees and back. Augeaus is already in portage. Parsing languages is hard without a grammar, so using an AST is the only way I would trust a program to modify a configuration file on any system. Augeaus is written in C, but has python bindings.
26
27	Support for remote editing on multiple computers simultaneously. It would be really nice if ventoo could be installed on one computer, then connect to several computers, parse their configuration files, then apply changes across all of them. If this is used along with ASTs then the configuration files wouldn't even have to be the same on each computer. I ask for help here, I don't know the best way to implement this. Maybe just scp to make the changes? I'm assuming that all the computers have the same root password and it would be used in an administration setting with many many computers. What's the best way to handle multiple computers with different configurations? Can we make this really slick and powerful? See implementation details below.
28
29	Parse the comments that generally appear above each configuration variable and display them to the user as documentation. Most Gentoo configuration files have comments that appear just above the option to configure, these can also be parsed into an AST. In the implementation section there are details on providing custom documentation.
30
31	Use the python to provide smart completion within the visual config editor. Eg: In /etc/conf.d/consolefont CONSOLEFONT is set to something listed in /usr/share/consolefonts Would be nice to have scripts what list and return possible options for variables like this. It could be as simple as hardcoded values, or dynamically listed values from python/bash/whatever scripts for each system and configuration file. Hopefully some of the existing code can be adapted to help with this.
32
33	Display config file changes as a diff file. Optionally highlighted with a 3^rd party program like meld, or library like gtksourceview. I say optionally because I think it's best to keep dependencies for a program like this to a bare minimum.
34
35	Integrate rc-update and a few eselect modules. Eg, select a kernel from a list.
36
37	Optionally provide hints in the ventoo modules about variable types. For example a path variable would prompt the user in an open dialogue. An enumeration with a dropdown box. Default to a plain text box.
38
39	Implementation Details
40
41	I am proposing to have separate modules manage the grammars for the configuration files. I am imagining that these modules would be stored in /usr/share/ventoo. Each module is defined as a grammar, code completion scripts, and variable/value documentation. The only required component of a module is the grammar so that Augeaus can parse the file and write it back. Code completion scripts return possible values to be entered into specific variables in configuration files. Example, if a variable asks for a specific device hard disk on the system then a code completion module would return ls /dev/sd* /dev/hd*.
42	Right now ventoo uses an html browser to display ventoo specific help. I propose we use this existing dependency for displaying the optional documentation mentioned above. If the parsed comments from a configuration file are not sufficient then the ventoo module maintainer could write additional documentation as an html file that ventoo could display as the user is editing that variable.
43	The fact that modules are completely separate from the core ventoo code means that a packages that have configuration files can provide ventoo with these module files and and visual configuration file editing support for themselves. Kind of like emacs mode use flags, packages add their own emacs modes.
44
45	If ventoo is editing several computers at the same time then things could get complicated when configuration files aren't the same for all of them. I think realistically it would be impossible to implement this properly over one summer, but room should definitely be made in the gui and in the code for future work here. This summer I imagine only implementing remote editing for a single computer at a time over scp and ssh and all the while adding comments about how to start off with multiple computers. The only hard part here is how to deal with different computers that have different configuration files and only some options need to be edited. How do we display this information intelligently to the user?
46
47	Deliverables
48
49	This is a list things that I promise to implement over the summer to pass the GsoC project.
50
51	*
52
53	Augeas grammars for most configuration files on a standard unix system.
54
55	*
56
57	Displaying important nodes in an AST
58
59	*
60
61	Writing ASTs back to files
62
63	*
64
65	Displaying diff generated from ventoo
66
67	*
68
69	Remote configuration of one other computer at a time
70
71	*
72
73	rc-update integration
74
75	*
76
77	Working documentation displayed in embedded html browser
78
79	*
80
81	Optional custom documentation
82
83	Timeline:
84
85	May 26^st to June 2^nd : Write Augeas grammars, start writing code to walk through ASTs
86	June 2^nd to June 9^th : Have a basic gui to display config file variables.
87	June 9^th to June 16^th : Make the GUI nicer, integrate rc-update, diff files
88	June 16^th to June 23^rd : Buffer zone, make sure everything is perfectly stable up to this point. If things take longer than expected use this week. If things go smoothly use this week to implement extras, like nice documentation modules and nice code completion modules. Make sure that the program can parse some standard config files, display the diff and write back. Complete midterm evaluations. At this point the application would be able to read/write ASTs and have a few ventoo modules.
89
90	By July 14^th make code completion scripts working. Make GUI handle complex config scripts, example, fstab. Fstab has rows and columns, each with different types therefore the GUI can't just be a linear list of variables, it has to be able to display a matrix with different types in each cell.
91
92	At this point there are 4 weeks left to fix bugs, write extensive and complete ventoo modules and ventoo documentation. Create ebuilds. Test on multiple systems. Edit other packages like cups or samba to have their own ventoo modules that could be enabled with a ventoo use flag. This region of time can also act as a buffer to make sure that there is enough time to complete everything on the above list so that ventoo useable in practice by an end user. If everything goes perfectly there may even be time to work on editing multiple computers at the same time.
93
94	There are a few points I especially want to get some input on.
95	Implementation language? This project is written python. I like python, but I like C and C++ more. I like to use glade with gtk-buidler but the current code has widgets hardcoded in.
96
97	Separate GUI?
98	Can we separate the GUI code from the internal workings of ventoo? Maybe others would like to add their own GUIs.
99
100	How important is it to keep old code?
101	I'm asking this because right now ventoo uses python to parse config files. It also has the GUI coded in with the project as the parsing. IMO these are both weaknesses but at the same time it kills me to rewrite what has already been done. I'd rather move forward keeping existing efforts.
102
103	I also want to point out that writing a GUI should never be a big deal...after all it's basically just putting widgets in a window. So long as the GUI is modular and isn't tied in too tightly with the inner workings of a program I don't mind doing it in python, glade or otherwise. I feel like this isn't my call so I'm asking the people who make these decisions to let me know what has the best chance of getting accepted.
104
105	Thanks.
106	- Christopher Harvey
107	Computer Science, 3^rd year, McGill University.