1 This is an idea from [[JoshTriplett]]. --[[Joey]]
3 Some uses of ikiwiki, such as for a bug-tracking system (BTS), move a bit away from the wiki end
4 of the spectrum, and toward storing structured data about a page or instead
7 For example, in a bug report you might want to choose a severity from a
8 list, enter a version number, and have a bug submitter or owner recorded,
9 etc. When editing online, it would be nice if these were separate fields on
10 the form, rather than the data being edited in the big edit form.
12 There's a tension here between remaining a wiki with human-editable source
13 files, containing freeform markup, and more structured data storage. I
14 think that it would be best to include the structured data in the page,
15 using a directive. Something like:
18 \[[data yaml="<arbitrary yaml here>"]]
21 As long as the position of the directive is not significant, it could be
22 stripped out when web editing, the yaml used to generate/populate form fields,
23 and then on save, the directive regenerated and inserted at top/bottom of
26 Josh thinks that yaml is probably a good choice, but the source could be a
27 `.yaml` file that contains no directives, and just yaml. An addition
28 complication in this scenario is, if the yaml included wiki page formatted content,
29 ikiwiki would have to guess or be told what markup language it used.
31 Either way, the yaml on the page would encode fields and their current content.
32 Information about data types would be encoded elsewhere, probably on a
33 parent page (using a separate directive). That way, all child pages could
34 be forced to have the same fields.
36 There would be some simple types like select, boolean, multiselect, string, wiki markup.
37 Probably lists of these (ie, list of strings). Possibly more complex data
40 It should also be possible for plugins to define new types, and the type
41 definitions should include validation of entered data, and how to prompt
42 the user for the data.
44 This seems conceptually straightforward, if possibly quite internally
45 complex to handle the more complicated types and validation.
47 One implementation wrinkle is how to build the html form. The editpage.tmpl
48 currently overrides the standard [[!cpan CGI::FormBuilder]] generated form,
49 which was done to make the edit page be laid out in a nice way. This,
50 however, means that new fields cannot be easily added to it using
51 [[!cpan CGI::FormBuilder]]. The attachment plugin uses the hack of bouilding
52 up html by hand and dumping it into the form via a template variable.
54 It would be nice if the type implementation code could just use
55 FormBuilder, since its automatic form generation, and nice field validation
56 model is a perfect match for structured data. But this problem with
57 editpage.tmpl would have to be sorted out to allow that.
61 * Pagespecs that can select pages with a field with a given value, etc.
62 This should use a pagespec function like field(fieldname, value). The
63 semantics of this will depend on the type of the field; text fields will
64 match value against the text, and link fields will check for a link
65 matching the pagespec value.
66 * The search plugin could allow searching for specific fields with specific
67 content. (xapian term search is a good fit).
71 [[tracking_bugs_with_dependencies]]
73 > I was also thinking about this for bug tracking. I'm not sure what
74 > sort of structured data is wanted in a page, so I decided to brainstorm
77 > * You just want the page to be pretty.
78 > * You want to access the data from another page. This would be almost like
79 > like a database lookup, or the OpenOffice Calc [VLookup](http://wiki.services.openoffice.org/wiki/Documentation/How_Tos/Calc:_VLOOKUP_function) function.
80 > * You want to make a pagespec depend upon the data. This could be used
81 > for dependancy tracking - you could match against pages listed as dependencies,
82 > rather than all pages linked from a given page.
84 >The first use case is handled by having a template in the page creation. You could
85 >have some type of form to edit the data, but that's just sugar on top of the template.
86 >If you were going to have a web form to edit the data, I can imagine a few ways to do it:
88 > * Have a special page type which gets compiled into the form. The page type would
89 > need to define the form as well as hold the stored data.
90 > * Have special directives that allow you to insert form elements into a normal page.
92 >I'm happy with template based page creation as a first pass...
94 >The second use case could be handled by a regular expression directive. eg:
96 > \[[regex spec="myBug" regex="Depends: ([^\s]+)"]]
98 > The directive would be replaced with the match from the regex on the 'myBug' page... or something.
100 >The third use case requires a pagespec function. One that matched a regex in the page might work.
101 >Otherwise, another option would be to annotate links with a type, and then check the type of links in
102 >a pagespec. e.g. you could have `depends` links and normal links.
104 >Anyway, I just wanted to list the thoughts. In none of these use cases is straight yaml or json the
105 >obvious answer. -- [[Will]]
107 >> Okie. I've had a play with this. A plugin is included inline below, but it is only a rough first pass to
108 >> get a feel for the design space.
110 >> The current design defines a new type of page - a 'form'. The type of page holds YAML data
111 >> defining a FormBuilder form. For example, if we add a file to the wiki source `test.form`:
116 comment: This is a test
120 >> The YAML content is a series of nested hashes. The outer hash is currently checked for two keys:
121 >> 'template', which specifies a parameter to pass to the FromBuilder as the template for the
122 >> form, and 'fields', which specifies the data for the fields on the form.
123 >> each 'field' is itself a hash. The keys and values are arguments to the formbuilder form method.
124 >> The most important one is 'value', which specifies the value of that field.
126 >> Using this, the plugin below can output a form when asked to generate HTML. The Formbuilder
127 >> arguments are sanitized (need a thorough security audit here - I'm sure I've missed a bunch of
128 >> holes). The form is generated with default values as supplied in the YAML data. It also has an
129 >> 'Update Form' button at the bottom.
131 >> The 'Update Form' button in the generated HTML submits changed values back to IkiWiki. The
132 >> plugin captures these new values, updates the YAML and writes it out again. The form is
133 >> validated when edited using this method. This method can only edit the values in the form.
134 >> You cannot add new fields this way.
136 >> It is still possible to edit the YAML directly using the 'edit' button. This allows adding new fields
137 >> to the form, or adding other formbuilder data to change how the form is displayed.
139 >> One final part of the plugin is a new pagespec function. `form_eq()` is a pagespec function that
140 >> takes two arguments (separated by a ','). The first argument is a field name, the second argument
141 >> a value for that field. The function matches forms (and not other page types) where the named
142 >> field exists and holds the value given in the second argument. For example:
144 \[[!inline pages="form_eq(age,15)" archive="yes"]]
146 >> will include a link to the page generated above.
148 >> Anyway, here is the plugin. As noted above this is only a preliminary, exploratory, attempt. -- [[Will]]
151 # Interpret YAML data to make a web form
152 package IkiWiki::Plugin::form;
156 use CGI::FormBuilder;
160 hook(type => "getsetup", id => "form", call => \&getsetup);
161 hook(type => "htmlize", id => "form", call => \&htmlize);
162 hook(type => "sessioncgi", id => "form", call => \&cgi_submit);
165 sub getsetup () { #{{{
169 rebuild => 1, # format plugin
173 sub makeFormFromYAML ($$$) { #{{{
175 my $YAMLString = shift;
180 eval q{use CGI::FormBuilder};
183 my ($dataHashRef) = YAML::Load($YAMLString);
185 my @fields = keys %{ $dataHashRef->{fields} };
187 unshift(@fields, 'do');
188 unshift(@fields, 'page');
189 unshift(@fields, 'rcsinfo');
191 # print STDERR "Fields: @fields\n";
195 $submittedPage = $q->param('page') if defined $q;
197 if (defined $q && defined $submittedPage && ! ($submittedPage eq $page)) {
198 error("Submitted page doensn't match current page: $page, $submittedPage");
201 error("Page not backed by file") unless defined $pagesources{$page};
202 my $file = $pagesources{$page};
206 if (defined $dataHashRef->{template}) {
207 $template = $dataHashRef->{template};
209 $template = "form.tmpl";
212 my $form = CGI::FormBuilder->new(
216 required => [qw{page}],
218 action => $config{cgiurl},
219 template => scalar IkiWiki::template_params($template),
220 wikiname => $config{wikiname},
227 $form->field(name => 'do', value => 'Update Form', required => 1, force => 1, type => 'hidden');
228 $form->field(name => 'page', value => $page, required => 1, force => 1, type => 'hidden');
229 $form->field(name => 'rcsinfo', value => IkiWiki::rcs_prepedit($file), required => 1, force => 0, type => 'hidden');
232 foreach my $x (qw{label type multiple value fieldset growable message other required validate cleanopts columns comment disabled linebreaks class}) {
236 while ( my ($name, $data) = each(%{ $dataHashRef->{fields} }) ) {
237 next if $name eq 'page';
238 next if $name eq 'rcsinfo';
240 while ( my ($key, $value) = each(%{ $data }) ) {
241 next unless $validkey{$key};
242 next if $key eq 'validate' && !($value =~ /^[\w\s]+$/);
244 # print STDERR "Adding to field $name: $key => $value\n";
245 $form->field(name => $name, $key => $value);
249 # IkiWiki::decode_form_utf8($form);
254 sub htmlize (@) { #{{{
256 my $content = $params{content};
257 my $page = $params{page};
259 my $form = makeFormFromYAML($page, $content, undef);
261 return $form->render(submit => 'Update Form');
264 sub cgi_submit ($$) { #{{{
268 my $do=$q->param('do');
269 return unless $do eq 'Update Form';
270 IkiWiki::decode_cgi_utf8($q);
274 eval q{use CGI::FormBuilder};
277 my $page = $q->param('page');
279 return unless exists $pagesources{$page};
281 return unless $pagesources{$page} =~ m/\.form$/ ;
283 return unless IkiWiki::check_canedit($page, $q, $session);
285 my $file = $pagesources{$page};
286 my $YAMLString = readfile(IkiWiki::srcfile($file));
287 my $form = makeFormFromYAML($page, $YAMLString, $q);
289 my ($dataHashRef) = YAML::Load($YAMLString);
291 if ($form->submitted eq 'Update Form' && $form->validate) {
293 #first update our data structure
295 while ( my ($name, $data) = each(%{ $dataHashRef->{fields} }) ) {
296 next if $name eq 'page';
297 next if $name eq 'rcs-data';
299 if (defined $q->param($name)) {
300 $data->{value} = $q->param($name);
304 # now write / commit the data
306 writefile($file, $config{srcdir}, YAML::Dump($dataHashRef));
308 my $message = "Web form submission";
310 IkiWiki::disable_commit_hook();
311 my $conflict=IkiWiki::rcs_commit($file, $message,
312 $form->field("rcsinfo"),
313 $session->param("name"), $ENV{REMOTE_ADDR});
314 IkiWiki::enable_commit_hook();
315 IkiWiki::rcs_update();
317 require IkiWiki::Render;
320 IkiWiki::redirect($q, "$config{url}/".htmlpage($page)."?updated");
323 error("Invalid data!");
329 package IkiWiki::PageSpec;
331 sub match_form_eq ($$;@) { #{{{
334 my @args=split(/,/, $argSet);
335 my $field=shift @args;
336 my $value=shift @args;
338 my $file = $IkiWiki::pagesources{$page};
340 if ($file !~ m/\.form$/) {
341 return IkiWiki::FailReason->new("page is not a form");
344 my $YAMLString = IkiWiki::readfile(IkiWiki::srcfile($file));
349 my ($dataHashRef) = YAML::Load($YAMLString);
351 if (! defined $dataHashRef->{fields}->{$field}) {
352 return IkiWiki::FailReason->new("field '$field' not defined in page");
355 my $formVal = $dataHashRef->{fields}->{$field}->{value};
357 if ($formVal eq $value) {
358 return IkiWiki::SuccessReason->new("field value matches");
360 return IkiWiki::FailReason->new("field value does not match");