Customized

To do this, Listing 5 first needs to include the newly created WTF snapshot module in the WTF source code of the widget_maker.go file. You will need a new import statement that drags in the code from Listing 6, as well as an additional case statement that calls the NewSettings() and NewWidget() functions from the Go snapshot package when the program is initialized. Listing 6 shows what goes on behind the scenes in the process. You need to copy Listing 6 to the modules/snapshot/ directory of the open source project, along with its counterparts in Listing 7 and Listing 8, before recompiling.

package app
import (
  //...
  "github.com/wtfutil/wtf/modules/snapshot"
  // ...
)
// MakeWidget creates and returns instances of widgets
func MakeWidget(
  // ...
  switch moduleConfig.UString("type", moduleName) {
  case "snapshot":
    // ...
    settings := snapshot.NewSettingsFromYAML(moduleName, moduleConfig, config)
    widget = snapshot.NewWidget(tviewApp, redrawChan, pages, settings)
    // ...
  }
  return widget
}

01 package snapshot
02 import (
03   "fmt"
04   "github.com/gdamore/tcell/v2"
05   "github.com/rivo/tview"
06   "github.com/wtfutil/wtf/utils"
07   "github.com/wtfutil/wtf/view"
08 )
09 type Widget struct {
10   view.ScrollableWidget
11   settings *Settings
12   err      error
13   links    []Link
14 }
15 func NewWidget(tviewApp *tview.Application, redrawChan chan bool, pages *tview.Pages, settings *Settings) *Widget {
16   widget := &Widget{
17     ScrollableWidget: view.NewScrollableWidget(tviewApp, redrawChan, pages, settings.Common),
18     settings: settings,
19   }
20   widget.SetRenderFunction(widget.Render)
21   widget.InitializeRefreshKeyboardControl(widget.Refresh)
22   widget.InitializeHelpTextKeyboardControl(widget.ShowHelp)
23   widget.SetKeyboardChar("j", widget.Next, "Select next item")
24   widget.SetKeyboardChar("k", widget.Prev, "Select previous item")
25   widget.SetKeyboardKey(tcell.KeyEnter, widget.openLink, "Open story in browser")
26   return widget
27 }
28 func (widget *Widget) Refresh() {
29   links, err := scrapeLinks()
30   widget.err = err
31   widget.links = links
32   widget.SetItemCount(len(widget.links))
33   widget.Render()
34 }
35 func (widget *Widget) Render() {
36   widget.Redraw(widget.content)
37 }
38 func (widget *Widget) content() (string, string, bool) {
39   title := "Programmier-Snapshot"
40   content := ""
41   for idx, link := range widget.links {
42     row := fmt.Sprintf(`[%s]%2d. %s`,
43       widget.RowColor(idx), idx+1,
44       tview.Escape(link.title),
45     )
46     content += utils.HighlightableHelper(widget.View, row, idx, len(link.title))
47   }
48   return title, content, false
49 }
50 func (widget *Widget) openLink() {
51   sel := widget.GetSelected()
52   if sel >= 0 && widget.links != nil && sel < len(widget.links) {
53     url := widget.links[sel].url
54     utils.OpenFile(url)
55   }
56 }

package snapshot
import (
  "github.com/olebedev/config"
  "github.com/wtfutil/wtf/cfg"
)
const (
  defaultFocusable = true
)
// Settings contains the settings for the snapshot view
type Settings struct {
  *cfg.Common
}
// NewSettingsFromYAML creates the settings for this module from a YAML file
func NewSettingsFromYAML(name string, ymlConfig *config.Config, globalConfig *config.Config) *Settings {
  snapshot := ymlConfig.UString("snapshot")
  settings := Settings{
    Common: cfg.NewCommonSettingsFromModule(name, snapshot, defaultFocusable, ymlConfig, globalConfig),
  }
  return &settings
}

01 package snapshot
02 import (
03   "errors"
04   "fmt"
05   "net/http"
06   "regexp"
07   "strings"
08   "github.com/PuerkitoBio/goquery"
09 )
10
11 type Link struct {
12   title string
13   url   string
14 }
15
16 func scrapeLinks() ([]Link, error) {
17   links := []Link{}
18   res, err := http.Get("https://perlmeister.com/art_eng.html")
19   if err != nil {
20     return links, err
21   }
22   defer res.Body.Close()
23   if res.StatusCode != 200 {
24     return links, errors.New("Fetch failed")
25   }
26   doc, err := goquery.NewDocumentFromReader(res.Body)
27   if err != nil {
28     return links, err
29   }
30
31   var maxHits = 5
32   daterx := regexp.MustCompile(`\d{4}/\d{3}`)
33
34   doc.Find("a").Each(func(i int, s *goquery.Selection) {
35     if maxHits > 0 {
36       link, _ := s.Attr("href")
37       if strings.Contains(link, "Issues") {
38         rs := daterx.FindStringSubmatch(link)
39         title := fmt.Sprintf("%s (%s)", s.Text(), rs[0])
40         links = append(links, Link{title: title, url: link})
41         maxHits--
42       }
43     }
44   })
45   return links, nil
46 }

The new snapshot widget on the right in Figure 6 is derived from the view.ScrollableWidget basic type, as shown in line 10 of Listing 6. This ensures that you can navigate to the widget and browse its content. The code in Listing 7 initializes the new widget with the YAML configuration data. As a result, the snapshot-specific Widget structure (Listing 6, line 9) can include additional YAML data afterwards, which is a no-op (no operation) in this case because the widget does not require any additional configuration. In addition to the YAML data, however, the Widget structure includes internal data in the form of the "Programming Snapshot" columns fetched from the web, along with their headings and URLs on the Linux Magazine site. Later on, Listing 8 defines the corresponding Link structure to hold these values starting in line 11.

The NewWidget() function in Listing 6 starting in line 15 creates the new snapshot widget in the WTF universe. It fills the Widget structure with the required content registers the widget with the renderer, which later draws it in the terminal UI. Lines 23 to 25 specify the keyboard functions that make the currently selected entry in the widget move up and down when you press K and J, while Enter selects the highlighted entry (along with the default browser action for the stored URL).

The Refresh() function starting in line 28 gets called whenever the terminal UI redraws the widget. Using scrapeLinks() in line 29, it fetches the links for current and past "Programming Snapshot" columns from the Perlmeister website, as detailed below in the web scraper in Listing 8, and breaks them down for displaying in a compact format for individual selection.

Triggered by the Render() command in Listing 6, the UI displays the current content of the snapshot widget on the screen. The content() function collects the content from line 38. It winds its way through the "Programming Snapshot" columns stored in the links instance variable and inserts them into the rows of the widget one by one with color highlighting.

Line 25 defines what happens when you press Enter after selecting a "Programming Snapshot" column, carried out by the openLink() function, which starts in line 50. Using the index number of the entry in question in sel, line 53 retrieves the URL for the entry, which is stored in the links data structure, and uses utils.OpenFile() to open it. This fires up the default web browser and tells it to display the contents of the article page on the Linux Magazine website.

Nothing really exciting is required in the YAML settings of the configuration file for the snapshot widget; only the standard stuff is processed. The widget does not have its own parameters, so Listing 7 only contains boilerplate code.

Data Hog

But how does the widget know which "Programming Snapshot" columns actually exist on the Linux Magazine website? To find out, the data grabber in Listing 8 scans the complete list of all "Programming Snapshots" published in the past 25 years on the Perlmeister.com site. The Go goquery scraper has an easy task with the simple HTML of the article links published under the URL defined in line 18. Its Find() function goes through all the links in the web document art_eng.html starting in line 34, only keeping track of the ones that have an Issues string in their path. These are typically links to "Programming Snapshot" articles on the Linux Magazine site.

Depending on the value defined in the variable maxHits (line 31), the function collects the URLs of a maximum of five articles, extracts the year and issue number of the publication from their paths, and appends them to the array of link structures in links. Each entry also features a title field containing the headline to be displayed in the selection along with the corresponding link to the article on the Linux Magazine website. Based on this list, the code uses the title fields of each element to generate the displayed list. When you press Enter to select an entry, the code grabs the url attribute of the entry and brings up the external browser for your reading pleasure.

Outlook

All done! Obviously, however, you can teach the WTF tool many more new tricks. It goes without saying that there are virtually no limits to what creative programmers can do with this tool.

The Author

Mike Schilli works as a software engineer in the San Francisco Bay Area, California. Each month in his column, which has been running since 1997, he researches practical applications of various programming languages. If you email him at mailto:mschilli@perlmeister.com he will gladly answer any questions.