## APL a Day #5: Indexing into Arrays

By now, clearly a shape and vector of values precisely determines an array. All APL functions operate over arrays; they describe new arrays in terms of its arguments. What about describing a sub-array of another array? Indexing provides a means of identifying specific elements of an array. Some people might call this addressing elements of the array.

Given an array `A` with shape `⍴A` and elements `,A`, a valid index `I` into array `A` is a vector whose length matches `⍴A`, stated formally as follows:

``````⍴I ←→ ⍴⍴A ⍝ Shape of Index matches rank of A
``````

An index describes the location of an element when arranged according to the shape. That is, an index encodes the position of an element according to the shape. Consider a matrix of shape `5 3` filled with the first 15 natural numbers:

``````    5 3⍴⍳15
0  1  2
3  4  5
6  7  8
9 10 11
12 13 14
⍳15
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
``````

Notice that the elements of `⍳15` correspond to their position in the element vector of `5 3⍴⍳15`, expressed notationally as ```⍳15 ←→ ,5 3⍴⍳15```. It feels awkwards to refer to the element in the third row and second column of a matrix as the 7th element of the matrix. True, the element in row 3 and column 2 does indeed correspond to the 7th element of the array as counted by its ravel (recall that the term ravel means the element vector), but no one wishes to do the necessary mental computation to make this happen. A key advantage to multi-dimensional arrays begins with the ability to reference elements relative to their arrangement according to shape, rather than positionally in the ravel.

Instead of referring to the 7th element in a matrix of shape `5 3`, the index `2 1` means the same thing. All indexing and counting in Co-dfns and this blog begins with zero, rather than 1. (A long standing debate concerning the origin of counting, be it 1 or 0, continues to gently ruffle programmer feathers within the APL community, but zero-based indexing holds the crown for most modern computer science concerns, and this series uses an index origin of 0.) Thus, the index `2 1` refers to the element at the third row and second column of the matrix.

All elements of an index must fall within the range described by the corresponding dimension of the shape of the array indexed. That is, the index `2 1` means something for an array of shape `5 3`, but refers to a non-existent element in an array of shape `2 2`. Such invalid indexes trigger INDEX ERROR in the APL interpreter. The following equivalence formalizes the requirement for elements of an index to fall within valid ranges, where `∧/` means “for all”:

``````∧/(0<I)∧I<⍴A ⍝ Index between 0 and shape of array indexed
``````

Certain APL functions allow an array and valid index into that array to describe the corresponding element indexed in the array. Writing `I⌷A` gives the scalar element of A indexed by `I`. To understand this relationship, the following picture helps:

``````┌→─────────┬─────────────┐
│┌→─┬──┬──┐│┌→──┬───┬───┐│
│↓0 │1 │2 ││↓0 0│0 1│0 2││
│├~→┼~→┼~→┤│├~─→┼~─→┼~─→┤│
││3 │4 │5 │││1 0│1 1│1 2││
│├~→┼~→┼~→┤│├~─→┼~─→┼~─→┤│
││6 │7 │8 │││2 0│2 1│2 2││
│├~→┼~→┼~→┤│├~─→┼~─→┼~─→┤│
││9 │10│11│││3 0│3 1│3 2││
│├~→┼~→┼~→┤│├~─→┼~─→┼~─→┤│
││12│13│14│││4 0│4 1│4 2││
│└~→┴~→┴~→┘↓└~─→┴~─→┴~─→┘↓
└─────────→┴────────────→┘
``````

The array above shows the correspondence between positions in the element vector and the indexes that refer to these positions in a ```5 3``` matrix. See how the elements are arranged by position in row-major order on the left. Notice the row-major order evident in the progression of the indexes on the right. The upper left hand corner of the matrix is `0 0` while the lower right hand corner will always be `(⍴A)-1`. In this case, the shape is `5 3` so `4 2≡(⍴A)-1`.

The power to generate the right hand side of the above picture rests firmly in your hands already. While `⍳N` serves as a nice method for generating natural numbers, the monadic `⍳` function actually accepts any valid shape as an argument. Given a shape `S`, the expression `⍳S` describes an array of shape `S` whose elements are the indexes corresponding to the element position. Thus, the above right hand side is a picture displayed by the APL session for `⍳5 3`:

``````    ⍳5 3
┌→──┬───┬───┐
↓0 0│0 1│0 2│
├~─→┼~─→┼~─→┤
│1 0│1 1│1 2│
├~─→┼~─→┼~─→┤
│2 0│2 1│2 2│
├~─→┼~─→┼~─→┤
│3 0│3 1│3 2│
├~─→┼~─→┼~─→┤
│4 0│4 1│4 2│
└~─→┴~─→┴~─→┘
``````

So, indexing into the array `5 3⍴⍳15` using `2 1` should give `7`:

``````    2 1⌷5 3⍴⍳15
7
``````

You now know how to reference any element in an array based on that element’s index into the array. Furthermore, a simple exploration of this reveals how to extract the Nth element from any array:

``````    N⌷,A ⍝ The Nth element of array A
``````

In the above example, it holds that `7⌷,5 3⍴⍳15 ←→ 2 1⌷5 3⍴⍳15`. Clearly, now, the expression `⍳S` degenerates into `⍳N` when `S` describes a vector. A vector’s indexes are natural numbers.

The next session with further explore the interesting relationship between an index and the element corresponding to it.

Please support my work via GitTip.

## Slackware DVD Boot Problems on Macbook

The other day a friend of mine wanted to see if I could install Slackware64 14.1 on a MacBook that he received. It was an older model, and the previous owner had wiped out the disks trying to get various Linuxen installed.

My friend already had a freshly minted copy of the DVD ready for us, but when we tried to boot it up, all we say was this message:

``````1.

2.

Select CD-ROM Boot Type:
``````

Of course, it didn’t help that the keyboard did not respond. Looking around, I noticed some advice on fixing this in various forums, but it didn’t quite work, because none of these options was available in building my own custom ISO image. I had also considered getting reFind installed on the system, but I realized that installing reFind on a MacBook without Mac OS X installed is really hard.

After some thinking and pondering about the message, I stared at the command to build a Slackware ISO for a while. It hit me then that the normal command used to build the ISO creates two boot paths, one for traditional systems, and one for EFI. Here’s the normal boot command:

``````mkisofs -o /tmp/slackware-dvd.iso \
-R -J -A "Slackware Install" \
-hide-rr-moved \
-v -d -N \
-sort isolinux/iso.sort \
-b isolinux/isolinux.bin \
-c isolinux/isolinux.boot \
-eltorito-alt-boot -no-emul-boot -eltorito-platform 0xEF \
-eltorito-boot isolinux/efiboot.img \
-m 'source' \
-V "SlackDVD" .
``````

The `eltorito` lines here cause the problem. Specifically, the MacBook doesn’t seem to be able to figure out how to boot from the CD given two different options.

The solution to this problem is easy: remove the alternative boot images and boot using the traditional methods. In that case, the new command to build the ISO looks like this:

``````mkisofs -o /tmp/slackware-dvd.iso \
-R -J -A "Slackware Install" \
-hide-rr-moved \
-v -d -N \
-sort isolinux/iso.sort \
-b isolinux/isolinux.bin \
-c isolinux/isolinux.boot \
-m 'source' \
-V "SlackDVD" .
``````

After using this new ISO, the MacBook builds and boots and installs without a hitch. Happy slacking!

Please consider supporting my work through GitTip.

## RetroShare 0.5.5c SlackBuild

I just submitted the RetroShare 0.5.5c SlackBuild. I recommend that all users of Slackware RetroShare upgrade. The approval should be coming shortly, so you should soon see it in the main SlackBuilds site.

## APL a Day #4: Arrays have elements

An array that only describes a box which may contain elements, but that does not contain any elements, does not mean much. While shapes help to clarify the APL language, when doing any work, they really only describe how to arrange the elements of an array. The monadic ravel function (written as the comma `,`) describes a vector of all the elements of an array taken from the array in row major order. Thus, the following property holds:

``````A ←→ (⍴A)⍴,A ⍝ A is the reshape of the elements of A with the
⍝ shape of A.
``````

That is, the shape of an array, together with its elements, fully describes an array.

Given the ravel of an array, how does the shape of an array describe the arrangement of those elements? The official computer science term for this is row major order. To understand how row major order works, consider a matrix, that is, an array of two dimensions. Supposing that the matrix has no non-zero dimensions, the shape of that matrix will be some positive integers N M. Since APL uses row major order, the N is said to be the number of rows in the matrix, while M is said to be the number of columns. Notice that the “most major” dimension appears first in the vector N M; the dimensions of an array are arranged in the array’s shape from most to least major. That is, the first element in the shape is the most major dimension, the second element the second most major, and so forth.

The monadic function “Index” (`⍳`) helps to illustrate these concepts. The notation `⍳N` where `N` is a natural number, describes a vector of the first `N` natural numbers.

``````    ⍳7
0 1 2 3 4 5 6
``````

The index function helps provide a set of sample values from which to describe arrays of various shapes. The following examples illustrate how the shape of an array arranges elements of an array:

``````    ⍬⍴⍳9   ⍝ An empty shape gives a scalar
0
3⍴⍳9   ⍝ A vector fills from the elements as needed
0 1 2
8⍴⍳4   ⍝ An array fills repeatedly from its elements
0 1 2 3 0 1 2 3
2 3⍴⍳9 ⍝ Matrices fill rows at a time
0 1 2
3 4 5
3 3⍴⍳4 ⍝ Matrices also fill repeatedly
0 1 2
3 0 1
2 3 0
``````

Notice that matrices fill from the elements repeatedly starting a row at a time. This explains the term row major. When arranging elements according to the shape, the first index of the most major dimension fills before the second most major index. In the examples above, the row was the most major dimension, so each row filled before the next row.

This process generalizes to any number of dimensions. To give an intuition to this, consider a 3-dimensional array with shape `2 3 3` fill from the elements `⍳4`.

``````    2 3 3⍴⍳4
0 1 2
3 0 1
2 3 0

1 2 3
0 1 2
3 0 1
``````

In this case, APL needs to represent a 3-dimensional array on a two-dimensional grid of characters. To do this, it arranges each sub-matrix as normal, but then separates each sub-matrix with a blank line. Thus, for a shape of `2 3 3`, APL displays such an array as two matrices separated by a blank line. Notice what happens if the shape changes to `3 2 3`.

``````    3 2 3⍴⍳4
0 1 2
3 0 1

2 3 0
1 2 3

0 1 2
3 0 1
``````

See how the elements continue to be arranged in the same way, but the size of the first dimension (the most major one) increased, while the second dimension size decreased. Now, instead of having two `3 3` matrices, APL displays three `2 3` matrices.

Play with using different elements in the array and using different shapes, with both the same and different elements. Gaining a good intuition for the concept of elements and shape together and how they work can take a little time. Don’t worry.

As a final note, the astute reader will realize that the number of elements given as the right argument to the reshape (`⍴`) function do not necessarily need to match the number of elements that the reshape function describes, as given in the left argument corresponding to the shape of the described array. Indeed, reshaping an array will continue to pull elements from the ravel of that array, as many as needed, repeatedly, without complaint. As a parting gift, spend some time pondering these properties, which speak more about this unique feature and others.

``````×/S ⍝ The product of the elements of shape S
⍝ also known as the number of elements in an array
⍝ of shape S

S⍴A ←→ S⍴,A ⍝ Reshaping any array is the same as reshaping its
⍝ ravel

(×/S)⍴A ←→ ,S⍴A ⍝ The ravel of a reshaped array is the same as
⍝ the reshape of that array with the product
⍝ of all the dimensions in the shape

S⍴A ←→ S⍴(×/S)⍴A ⍝ Reshaping an array A is the same as reshaping
⍝ the ravel of A replicated to match the product
⍝ of the dimensions of the shape
``````

## Slackware 14.1: Why take a closer look?

Slackware Linux remains among the top 20 most “visitied” distributions on the DistroWatch website, though such numbers mean next to nothing when determining distribution worthiness. As the oldest surviving Linux distribution, Slackware possesses remarkable staying power. Nonetheless, while most respect the distribution, Slackware bears a fearsome reputation. Linux newbies frequently associate Slackware with doing it the hard way, or going it old school. This fearsome reputation results from a poor mix of facts and a healthy dose of losing sight of the forest for the trees. Even so, does Slackware deserve any attention today? It may have once been the most popular distribution, but the modern Linux user faces a sea of potential choices each one of varying quality and attractiveness.

From the perspective of a heavy Slackware user, this venerable distribution satisfies a need and delivers a value both larger and broader than its largely inaccurate reputation implies. Do not let Slackware’s reputation deter you from considering it seriously. Indeed, the reasons Slackers so faithfully appreciate and use Slackware for so many things resonate with the desires of an obvious and large group of Linux users and even beyond.

## Why use Slackware?

Consider the following compelling Slackware virtues, built on the perspective of doing real work on a daily basis with a workstation and one or two servers.

Slackware is Stable unlike any other. Distributions throw around the “Stable” label all too freely. What does this actually mean? Most distributions slap the term stable to a release when it is “good enough” for general consumption. Those that have a more enterprise oriented mind define stability in terms of the length of support or the longevity of the platform. Indeed, RedHat creates value by creating stability amidst the distribution chaos that is Linux; by allowing companies to anchor their businesses on technology that will exist in supported form for a fixed length of time, RedHat delivers confidence. Slackware does something different. Slackware does provide long support cycles (sometimes even longer than RedHat) for its releases, but more importantly, the releases themselves each contain the best quality that the Linux community has to offer at a given point. Each version delivers a total package that will prove to work well together and as a platform, work at the best levels possible, and do so reliably. Slackware combines, as no other distribution does, the goals of modern software and working software. Perhaps more so than any other distribution, Slackware just works, as intended, as described, release after release.

Slackware is predictable. Change is good; rampant chaos is not. Slackware continues to release up-to-date software that meets the modern needs of users, but it doesn’t fundamentally alter itself without good cause. When you learn Slackware, you know what to expect with each release, and when the time comes to deviate from the norm, these deviations receive prominent attention in the CHANGES_AND_HINTS.TXT file. While the components shift with the times, the fundamental glue, the invisible spirit of Slack that binds together the distribution to make it Slackware, these give predictable, reliable performance year after year. Moreover, because Slackware releases roughly yearly “when it is ready,” the distribution feels more solid in use, and less flighty; you simply don’t upgrade as often or with such a sense of urgency.

Slackware is easy to use. Ah, the controversial one. Slackware lists simplicity and stability as two driving goals. This simplicity makes the system easy to maintain and use. Slackware avoids needless complication. A simple set of skills suffices to administer a Slackware system successfully; these skills transfer from one task to the other. Everything from system upgrades to basic everyday configuration gets handled in similar fashion, a simple way. Slackware makes building packages, updating a kernel, installing custom software, updating software versions, configuring and loading X, setting up the network, configuring services, &c. easier and less complicated than other systems. It does so largely by getting out of the way.

Slackware is flexible. Distributions like Debian have many attractive qualities, but they often deliver these qualities by sacrificing flexibility. Debian can do some remarkable things with its package system, but you lose the ability to really work with and set up the packages and versions in a way that fits your needs. Slackware puts flexibility right out front, keeping with the KISS principle and letting you do pretty much whatever you need to do. Take the installer, for instance. Because the installer is so simple, and the process so straightforward, custom installation processes to support all sorts of needs can happen without complex infrastructure support. The system installers of other distributions might provide many features, but if your desired configuration and goals do not fit within their predefined set of actions, adding those features takes more work than just doing it without modifying the installer on Slackware. Because of the simplicity, you have a great deal of confidence in your actions, and what those actions will do to the rest of the system.

Slackware is Modern. Unlike many other distributions which provide long release cycles, predictable stability, and high reliability, Slackware keeps up with software, ensuring that the software is both stable and modern. Even better, Slackware does not get in the way of the user, so the user can easily keep an installation up to date above and beyond a conservative choice made by the distribution. Slackware users can stay as modern as desired without breaking the paradigm of the distribution.

Slackware is effective. The real reason Slackware sticks around? Slackware is effective. Slackware gets the job done, it gets it done well, and it gets it done without undue pain. Slackware isn’t a toy, it’s a serious tool for getting Real Work ™ done. It’s also a place of freedom and possibilities. Combine these together and you get an effective, hardy distribution that stands the test of time.

Slackware avoids needless patches. Slackware delivers software without undo modifications. This pleases developers and users alike: the software works the way the authors wrote it. This sort of vanilla purity feeds back into the flexibility mentioned above. Unmodified sources make things easier, and generally more stable.

## Who is Slackware for? Who should avoid it?

So, who might like Slackware? Who should consider it? Anyone who identifies more with UNIX practices than with Mac or Windows should take a close look at Slackware. Slackware is the most UNIX-like distribution, and it looks to remain that way. If you are tired of surprises and just want a workhorse distribution that reliably gets the job done well and will work for years and years, check out Slackware. If you enjoy the hacker ethos and being able to get into things and build and work on a distribution, Slackware is as accessible as any, and provides a convenient platform for such exploration and discovery.

I hesitate to suggest that anyone should avoid Slackware, because who knows whether they might find some unknown delight through using it? On the other hand, Slackware clearly does not aim to service a certain kind of customer. It does not pretend to be Windows or Mac. Users looking to naively and ignorantly use Linux as just a “better Windows” will probably find Slackware hard and unfriendly. Slackware could also bore the tweaker, or anyone else who enjoys the excitement of rapid releases and risky software. It’s going to work, and it’s going to be ready when it is ready. While you can run the -Current development if you want, the Slackware way doesn’t mesh as well with the twiddling tweaker as it does with the old school hacker.

Finally, if you can’t tell the difference between easy to use and “Windows user-friendly”, Slackware will either change your thinking, or you will not like it.

## Examining the latest 14.1 release

Because Slackware stays so remarkably similar from release to release – a good thing — reviewing the whole installation process, and all the software and so forth doesn’t make much sense. Suffice to say that it worked well, as intended, as documented, and without any hitches. What’s cool about the 14.1 release are the hidden gems that take a little polish to reveal.

SSD Caching. Bcache recently went into mainline kernels. The 3.10 kernel release has the bcache system in it, making it possible to use the SSD as a cache. The catch? Bcache doesn’t make it easy to add caching after installation. No other distribution I am aware of at this time has a stable release with the capacity to do a bcache install from the beginning. Out of the box, neither does Slackware. However, a custom initrd, possibly a slight modification on the initrd package, the bcache tools, and a rebuild installation media are all you need. Now, novices should not attempt this unless they want to learn more than they intended. However, if you’ve wanted to have access to SSD caching and wanted a clean, smooth way to do this, Slackware’s system is flexible and easy to use enough that you can make this happen in Slackware, without waiting on other distributions to get support for this into their installers.

Modern software. This release maintains a great set of modern software releases, and the Dropline Gnome project recently released an alpha version of Gnome 3.10 for Slackware 14.1. Now is a great time to trade up to a distribution that is both up to date and stable. Keep in mind that Slackware easily enables the upgrading of software even if Slackware doesn’t ship with it by default.

## Conclusion

While Slackware does not try to satisfy all types of Linux users, Slackware will satisfy a great deal of them who take the plunge. Slackware manages to consistently release stable, up to date versions while still keeping to the UNIX and old school hacker ethos. It remains simple and easy to use, and this latest release adds some cool new features for those who are looking to take advantage of some of the newest things happening in the Linux world.

## APL a Day #3: Arrays have Shape

Arrays arrange data into a certain rectangular shape. In APL, the shape of an array is a vector — yes, in APL, shapes are also arrays — that describes the number and size of layout in terms of dimensions. All arrays have a shape; if `A` is an array then `⍴A` is its shape. The function `⍴` is a primitive, that is, a built-in, ambivalent function. When used monadically, as in `⍴A`, this describes the shape of `A`. The expression `⍴A` describes a vector of natural numbers each corresponding to the size of one of the dimensions of `A`. Consider these examples:

``````⍬ ←→ ⍴S ←→ ⍴7 ⍝ Shape of a scalar is the empty vector
⍝ Scalars have no dimensions
,N ←→ ⍴V      ⍝ Vectors have one dimension
⍝ N is the number of elements in the vector
⍝ Note: ,N is a vector whose only element is N
,3 ←→ ⍴0 1 2  ⍝ An example shape of a vector (1 2 3)
``````

Consider the following:

``````0 1 2
3 4 5
6 7 8
``````

The above matrix contains 9 elements arranged into 3 rows and 3 columns. It therefore has two dimensions, first the rows, second the columns. To those with computer science background, the shape of an array arranges its elements (also called its ravel) in row-major order. The above array therefore has the shape `3 3`, since it has two dimensions, and the size of each of these dimensions is 3.

APL only permits directly writing scalars and vectors. One cannot write the above matrix literally into an APL session. Instead, APL allows the use of any array to describe a new array with a different shape. The dyadic use of `⍴` describes a new array whose shape is the left argument to `⍴` and whose elements come from its right argument. Consider one way to write the above matrix:

``````3 3⍴0 1 2 3 4 5 6 7 8
``````

Note that the following properties hold:

``````S ←→ ⍴S⍴A   ⍝ The shape of an array reshaped with S is S
A ←→ (⍴A)⍴A ⍝ An array reshaped by its own shape is itself
``````

The expression `⍴⍴A` describes the rank of `A`. The rank of an array is the number of its dimensions. In APL, the rank of an array is a vector of one element containing the count of that array’s dimensions. The following interesting little property falls out from this:

``````,1 ←→ ⍴⍴⍴A ⍝ The shape of a rank is 1
⍝ Or, the rank of a shape is 1
``````

APL’s treatment of rank speaks to its philosophy. Firstly, notice how the shape and reshape functions use the same symbol `⍴`. APL intentionally groups common operations together when possible. Most symbols in APL map to two functions, a monadic and dyadic version, each of which relates to the other in some way. APL primitives also apply to many different areas; that is, primitives have more than one application. Shape, for instance, serves not only as shape, but also provides the Rank function. APL does not provide a separate built-in function for rank, since shape subsumes rank and does so more generally. Finally, notice that arrays themselves can encode their own properties such as shape. The useful generality of arrays can astound even practiced APLers. APL can describe itself, and future sessions describe properties of APL using APL, as done here. In part, APL’s power comes from its ability to manipulate directly in code what other languages leave static, not unlike the power Lisp languages afford through their code-data duality. Remember that APL began as a notation for mathematical discourse, and as such, few languages match its concision and descriptive power.

The shapes of arrays give a great deal of information, especially when attempting to understand the language of APL itself, or when reasoning about APL code. Introducing Shape early enables future topics to leverage shape as a method for describing and detailing concepts clearly and concisely, which otherwise might seem arbitrary. Keep shape in mind when reading APL code, as the flow of shapes through a program speaks volumes.

Arrays are more than their shapes; they are shapes and elements. Arguably, the elements of an array aid learning less than the shape of an array, but to understand arrays is to understand shapes and their relation to elements. Next time should complete the treatment of arrays, and thus shall conclude the two fundamental concepts of APL: functions and arrays.