I guess if you keep looking for these you’ll find them. This one is somewhere in Atlanta Georgia and popped up in the Google Earth backgrounds for my browser.
btw: I’m still chunking away at the Shape Compositor but have hit some snags (actually writer’s block) and so nothing to report.
I’m reading the McCullough Wright Brothers book and so looked up Kitty Hawk on Google Earth. The Memorial to the Wrights surprised me, remarkably derived shape. I’ve enhanced the contrast to make it clearer.
Someone had fun designing this!
Here’s a little more click bait, a composite shape produced in prototype “compositor”.
This could be a nice repeating element in a drawing, I think. The asymmetry (within an overall symmetry) seems to create a nice balance, not too regular and mathematical, but also clearly based on math.
I reported yesterday on my progress in getting started on the shape compositor, which takes simple shapes from the shape generator (or a few built-in shapes) and composites them, with various transformations and replications to create a more complex shape. One major todo I listed was to add the clipping process and this went smoother than I expected so I can show what this does.
But first, as usual, I’ll throw in a teaser graphic to induce you, Dear Reader, to follow the rest of the explanation.
The clipping I added (through a rather complicated set of UI controls) allows using either a simple circle or one of the shapes, positioned at the center of the mirrored shapes, and with a controllable radius. So let’s look at the simplest versions of clipping:
The first figure, on the left, shows the composite shape with the clipping circle shown in light-blue (my standard color for “construction lines”) without any clipping. Then the middle figure shows the effect of clipping the composite shape “outside” the clipping circle and the rightmost figure shows the effect of clipping “inside” the circle.
Now the effect of this clipping (either one) is to leave open shapes which won’t work in the coloring program so, at minimum, we’d want to draw the circle to close the shapes, as seen below:
So we have closed shapes now, but there is a different way we can approach drawing the clipping circle and that is letting the clip circle be clipped by the shapes, as seen below:
So in the figure above the left shape has the clipping circle drawn “outside” the shapes and on the left it’s drawn “inside” the shapes.
So with just these clip options we can make some interesting composite shapes, but we can make it a bit more interesting, with the same clipping options, but using a different shape as per a few of the examples below:
So with a few simple shapes to mirror and a few more shapes to clip we can create all sorts of compound shapes that may be more interesting to use in the coloring drawings.
So with this step, at least in experimental coding form, completed it’s time to move on to rotate and repeat shapes around a path (initially just a circle) and in conjunction with clipping. But also to truly do compositing I need to allow more combinations, i.e. one set (more than all the same) shapes to be mirrored surrounded by rotating shapes with more complex clipping.
To do that I have to first figure out how I can reduce all these options to a “language” as trying all the combinations just through UI options is going to get very complicated. I have a few ideas about the structure of that language (based on my success (and shortcomings) of tieflume as a shape generation language). So that will be a challenge to figure out but it’s the essential part of the shape compositor. And, of course, the composite shapes need to go into shape library with some consistency with the simple shapes so when it comes time to render either type of shape into a drawing that will all go smoothly.
So stay tuned, this is evolving and hopefully will get more interesting.
I’ve been preoccupied with a visit from relative and so had little time to work; I need to have lots of private/quiet time to be able to focus and so haven’t had that, but I dabbled at bit and am now getting back in the swing.
Teaser image to hold your interest to read the post about how this is produced:
I have four components to the coloring system I’m building: 1) the coloring program itself, currently fairly advanced but I only have outline drawings I find elsewhere and, most often, low resolution and so a bit troublesome to color; eventually this program will be tuned to work with my own drawings, 2) a shape “generator”, which is currently based on parametric equations and an interactive approach that allows RSE (Rapid Shape Experimentation) – the interesting shapes from this go into a shape library (now in the form of the “history” my recent posts have described), 3) a shape “compositor” which can combine the simple shapes into more complex shapes – this is what I’m beginning to explore in this post, and, 4) the actual coloring drawing that will be created from various construction paths (and symmetries) as a layout, then filled in with either the simple or composite shapes.
Underlying these components (at least three of them) is the “shape library” which is more than simple bitmaps of the shape itself, but the “language” definition of them so they can be rendered, at arbitrary resolution, to be combined into composite shapes or drawings. It’s been challenging to think of exactly what functionality the shape library requires so I’ve (recently) built a surrogate or proof-of-concept (experimental) version as the “history” in the shape generator.
So partly to begin to understand (largely through vague ideas (not fully baked designs) and experiments) the requirements for both the shape library and the compositor itself I’ve started some work on it – not quite up to the level of a POC, more just as an experimental testbed to try some ideas and then generalize these experiments into a design and actually build that. So I’ve advanced from the purely hardcoded version I previously reported (the placement of multiple shapes (specifically “stars”) around an arc) to a new program that I am calling the v0.1 of the compositor itself. I did, with a fair amount of hassles, make some headway on this during the limit work time I had during my recent guest’s visit.
The place I started with, knowing I’d want this feature in the compositor is mirroring a simple shape, either vertically or horizontal or both, and then later adding some clipping (now working on) via simple shapes or the actual shapes from the generator. Thus combining shapes together to create more complex and interesting shapes.
So here’s what I mean by this, starting with a tieflume shape (below):
which has its “canonical” rendering in this orientation (actually 90° rotated from what the parametric equations draw in standard Cartesian space). This shape is a “family” in that its basic shape is retained by it can vary across a range of:
So using this shape we can create the following “mirrors”:
(the colors are arbitrary, just to help me in programming (with many mistakes) to identify the left, right, top, bottom instances of the shapes)
You can this is a “mirror”, not a rotation when you use an asymmetric shape as shown below:
You could rotate the orange shape to the red shape, but rotating again to the blue shape’s location would put the “curlicue” on the top, not the bottom as the mirror does.
An additional option to the mirroring is to determine whether the shapes overlap or are far apart, like this:
which then allows the possibility of creating an interesting composite shape from these simple shapes.
So even with just this simple feature in compositor it should be apparent how more interesting shapes can be created.
So where do I go from here, while still just in the “experimental” mode (vs actually more careful coding of a POC or even the first version).
First I want to add clipping (hopefully a post on that some) so I create something like a “flower”, with one set of “petals” on top of a second set (and different) petals, with a “center” that looks more like a center. Clipping will be challenging since, at least in .Net graphics, it is applied when rendering the overall drawing, but, AFAIK, I can’t actually “clip” a GraphicsPath with another GraphicsPath, select the parts I want and create a new GraphicsPath, like the example below:
When I fill the left ellipse with the right ellipse as a clip region (actually the region is the entire drawing space with a “hole” inside it where the second ellipse is), thus yielding the pink region. Fine, this would render correctly but I actually want the “outline” of the pink area (in highest resolution) so I can use it as a shape. Since the shapes are arbitrary the math necessary to compute the GraphicsPath looks really hard I suspect I’ll just use some tricks, basically render the shape to small line segments in very high resolution (like 4K) and then create the path via code tricks, not math.
After clipping I want the flexibility to make a composite not just of a single shape (repeated) but different shapes and also multiple placements (not just four instances of the shape). And instead of mirror I want to do a rotation (along the simple path of a circle) with various alterations of shape (say every other “star” in a 12-fold rotation is either a six-point larger and eight-point smaller, or, say 5 equally space large 5-point stars, with three smaller five point stars in between (and probably along a different circle radius).
And, of course, a much large library of shapes to work with.
And, ???, but undoubtedly other features I’ll discover as I do my coding experiments.
And all of this, again, wants to be controlled with a “language”, not just a bunch of stuff in the UI (already getting cluttered in my “experiment”, given the significant number of options just a few compositing tricks can require), but also so the composite shape can easily be changed by a few tweaks to the “program”. But in this case, unlike my tieflume “language” I’ll need a way to express inter-relationships (like make a path for placing shapes bigger, mean other placement paths adjust accordingly).
And then as a really interesting programming challenge instead of just distributing shapes around some simple reference path (simplest being circle, but might be other regular polygons) instead use a more geometrically complex tieflume shape (like the triangle with the curlicue) and distribute shapes along its perimeter with various spacing options (not just equal angle like a rotation, but also equidistant along the perimeter, then having to deduce the tangent to determine the rotation).
IOW, even with the still-not-very-specific specifications for the shape compositor I have in my head (and wishlist in notes) this is going to be a major development effort.
So stay tuned as this progresses because I believe it will produce some interesting results that would be difficult to achieve through other means (other than the visual cleverness of a skilled illustrator, which I’m not, so I write code to make complex drawings, not actually draw them).
It’s a very small step forward but a critical one as I’ve switched gears, yet again, on the part of my system I want to develop a bit. To review I have the shape generator on one end (upstream) and the coloring program on the other end (downstream). I need to create shapes (now multiple ways), feed those into the compositor to may more interesting complex shapes, and feed those into the drawing composer so I have an original drawing to do in the coloring program (and post for you, Dear Reader, to do with as you wish).
So here it is:
Big deal, some stars in a circle. But it is a step forward to start on a POC for the shape compsitor. The stars, now filled instead of outlined, are produced from the shape selection and configuration process, in this case a “builtin” shape, but the shape process can now be applied by shapes from the shape generator tieflume programs. The stars have multiple parameters that can adjusted as you wish, then placed in a configuration, in this case wrapped around a circle with no clipping (which adds all sorts of interesting possibilities). So this test program, literally test program (not even POC) can also build something like:
(you get to figure out the contradictory symmetries in this one).
Big whoop, still nothing interesting, but the point is the compositor is going to take various objects, use them with to repeat along some figure (a circle is simple, a bit trickier to put this around a heart), then add other figures (as either clip area or drawing area) and end up with something that is far more complex than the parts, AND, hopefully visually interesting (that’s up to the user, I’ll just try to build the tools).
Now a circle of stars is not much but it’s a bigger. That’s the nice thing about code – get this to work and it’s easy to extend to something far more complex, and, hopefully, more visually interesting. For instance, what if instead of the circle to place the stars (and properly rotate them, the thing that stumped me, for a while) you use multiple shape around the perimeter of a shape that is itself complex (say a heart). Once I get more software done I’m again focusing on something you can easily change (e.g. three different shapes around an unrelated shape) to finally, with foresight plan, or by luck stumble onto that is interesting. THEN place that shape into some complex pattern of a final mandala-like drawing.
So this is a step, trivial to some readers, but eventually I hope way more complex and non-obvious and also interesting.
The math I stumbled on getting this, while simple, was a problem. Like anyone who’s had analytical geometry, we’re very oriented to to +y is “up”, -x is “left” and so forth (all of the trig in this based on that). BUT, graphics, at least in Windows is counter-intuitive (in a trivial, but to me confusing as my developed intuition has trouble recognizing) that y increases by going down (in essence the fourth quadrant of the typical math/trig notion) and thus mirrored. So what I had to do was stay in -1…+1 as long as possible, then translate to the pixel coordinates to draw the final result. If I’d never know the usual Cartesian coordinate system, then perhaps it would be simple, but I constantly finding myself having to think with a mirror image, e.g. a “positive” (counterclockwise) rotation of 18° is actually equivalent to a “negative” (clockwise rotation of -72º), or really a 90º total rotation, but in five-fold symmetry very different.
Already, with my testing of the shape generator and my history file (past couple of posts) I have 33 shapes (many of which are families) to attempt to use, like:
with this shape:
placed around its perimeter with the long axis of this shape perpendicular to the path of the first shape, (weird and uninteresting, but again people with more design sense that me might make something more interesting.)
So that something I’m looking for, plus, btw, a lot of types of “distortions” that can be applied to the shape, such as:
So stayed tuned until I’ve got some of this working so my images will be real products of the shape compositor instead partial (and simple) results produced with crude prototypes, PhotoShop, and imagination.
It’s been a long slog, much longer than I expected, but now I have something working in terms of a real (and persistent) history of saved shapes (and their parameters and tieflume code to create) in my “blob” (compositeFile). I managed to install all the various history options and get a really simple “inspector” (needed to confirm everything I was saving in a big pile of internal objects was correct, as well as the first real verification of the RLE compression of 514×514 shape renderings.)
So that went well but boy, coding bogged down when I returned to “finishing” (at least enough) the compositeFile. Day after day with slow progress, but finally yesterday I got there. So I completed coding the integration yesterday with a very sketchy inspector. So today I reorganized the UI of the inspector and, very importantly, actually started showing thumbnails (128×128) in “album pages”, eventually with both a next/prev page working, but more usefully a cache of the thumbnails, after their first use.
I had no idea what to expect about performance. It’s unbelievable all the stuff that has to happen to get a thumbnail: 1) find all the blocks the blob is stored in in the composite file (with lots of verification nothing got broken), 2) copy data from blocks to an internal buffer (of variable size), 3) pass that data back to inspector, 4) who then restores the bitmap from the RLE (yet another copying of all the data), 5) then generates the thumbnail to display on the album page, and, 6) finally to cache the bitmaps so I don’t have to repeat all that when paging through the album to previously accessed pages. Fortunately generating a new page (18 shapes, 6 x 3 in a fairly big form and its picturebox) wasn’t that bad, about 3 seconds at fastest, 5 seconds to slowest, under a second when using the cache.
So the mechanism for recording history seems to work OK (I still don’t totally trust it, more testing will tell the whole story) and at least a simple way of finding the information again is available. The sequential browsing through album pages, however, is too slow and tedious to use once the history has thousands, even tens of thousands of shapes, but now that I have this much I can begin to focus on that problem, which was part of the point of this whole exercise.
So I’ve got a few things to do, at minimum:
Now this whole story doesn’t put up any pretty pictures on the screen for you to see but at least now I have a much better way to record those pretty pictures. I did a poor job of recording enough information in my OneNote log so I have some interesting shapes there I can’t figure out how to generate again (but now I can experiment with it, good old RSE) long enough to maybe stumble on them again.
And while grinding (and it has been a grind) on all this I keep thinking more and more about the shape compositor (which takes multiple simple shapes to make a more complicated one that probably can’t (certainly easily) be generated by the shape creator (it’s take it to generate lots of interesting simple shapes to use in more complex situations). And in the thinking I’m doing (much faster and wide ranging than trying to code the ideas I’ve already had) I realize this method of shape generator may not be sufficient so I’m thinking of how I might generate other types of shapes in yet another component.
So all this, shape generator shapes, composited shapes, and computed shapes is just to send enough stuff to the drawing composer, which is why I need the shape library and efficient access to it.
So despite hitting this major milestone I have a long way to go before I’ll be showing any of my generated mandalas (and giving them to you). But progress is progress and each time I get significant new functionality I usually think of even more things to do.
So stayed tuned, Dear Reader, this is all going somewhere, even with the gaps in my posting as I’m buried in 16 hour days turning ideas into working code.
Building the code for “container” (compositeFile) has been a lot slower slog than I anticipated, but I might be seeing the end. These 16-hour sessions are draining. Normally I’m more of a night-owl, but I do tend to see things more clearly in the morning (as now).
So I have the blob create, delete and read coded (almost entirely) with all the tons of integrity checking but I don’t know how robust it really is. So I need to build a stress test (massive numbers of creates, deletes, and reads, making sure to hit some corner cases (data exactly fills fixed number of blocks, data is +1 beyond filling, so creating shortest possible final block, creation that exceeds available free blocks, and so forth). Until I build that I won’t know exactly how well this is working.
But I need to try this on larger files than the typical test files I used, so I need automation to confirm the file is correct, so I’m looking at an “integrity” checker (already have a debugging version of compositeFile, which doesn’t use its objects and more directly accesses the internals of the file, thus representing a “independent check” (i.e. errors in the many classes of compositeFile itself would be bypassed by an independent way of analyzing the file, based on design specs, not code). This is a bit of non-trivial exercise so I’m somewhat reluctant to do it before attempting integration with shape generator and its existing non-persistent history, but I know once I do the integration I’ll start to use the history and so any bugs in it might cause terrible data loss of all the stuff I put in the history in “real” use.
Hopefully these parts will go smoothly and I can soon report the milestone that history file is working and then I can emphasize more its “browser”, i.e. how to access (smoothly and quickly) all the huge amount of data in the history file, which is partly the point, as that experience will allow me to better understand and define the requirements of the shape library, which is the point of this whole exercise.
Once I have the shape library (might even just use history) and the parametric equation rendering code completely refactored to be an engine (completely detached from UI) then I could actually start a POC of the shape compositor, which is too vague an idea, without getting some POC, to really decide other aspects of how the shape library would work.
I also am adding (mentally, as putative design) another kind of shape generator as I suspect certain types of shapes may be hard to create with the parametric equation approach and I want to be able to have a wide range of shapes to feed to the drawing composer (also still a very rough idea, I’d like to do some sort of POC for that so I can see if I’m on the right track for all of this.
So plenty to do, so I need to get out of this seemingly endless tunnel of building the compositeFile, which got fairly bogged down, but as it evolved also: a) requires a lot of refactoring for cleaner code (which I’ll need for future development when all the knowledge of its internal is flushed from my mental cache (when I finally take a break), and, b) there are a lot of dangling loose ends, partially implemented in current code, such as:
So a lot still to do on compositeFile, probably another week of slogging down in the mud, but I need to get out of that slog and get some fresh air to actually use the current, minimal (and still not verified) compositeFile so I can figure out what additional requirements I’ll need for it.
Incremental (vs designed) development has its faults, but given where I’ve already gotten to, incrementally, I seriously doubt I could have anticipated all these things and done a complete up-front design anyway, so I pay the price (refactoring, plus some yucky code still left) of figuring out what I need as I’m building it.
Stay tuned, hopefully a milestone to report soon. And then maybe some pictures instead of all these words.
I was hoping I could have done a post, at least a day ago, reporting a milestone – the full implementation of history (as POC for shape library) in the shape generator. But, alas, it was not to be. It’s been a slow slog trying to get this done.
I made great progress at first. I built all the UI and hooks into shape generator for history, even having to do the underlying stuff of the Run-Length Encoding to compress the shapes to save. That took less than a day. But I was merely: a) building a temporary XML to represent the history (should be persistent, but exactly how became more complicated question than I expected), and, b) just saving the shapes themselves in an in-memory heap so I could find them again. Again that went so smooth I was able, with a late night push, to even build a fairly simple history “browser”. That allowed me to get an idea, esp. as I’m not doing any I/O to get the shapes, of how fast the browser would be (not bad, but slow enough the “real” browser will have to do a lot of caching and have smarter ways to handle a very large number of shapes.
So buoyed by that quick success I thought I’d bang out the needed bits in my “container” (composileFile classes) and then integrate with the shape generator (have the exact hook in it I need). But then things really bogged down. So why?
With my architect hat on it takes less than an hour to setup the technical requirements for the compositeFile (I’d already done some of it but now actually putting some data in it, getting it back, and maintaining its integrity (really critical!) turned out to be much harder than I expected. In a real professional software effort no one would proceed with coding with such a skimpy definition that the (lousy) architect did. Or, at the very least now the person with designer hat on would flesh out the architect’s definition with some very detailed design of all the classes that would be needed. Now, in fact, this is still somewhat a methodological issue – should the architect have to go to that level of detail or is the detail (that the programmers will need, fairly precise definition of classes, all their methods and properties, and the functional requirements)? In some cases that is the job of the architect (and some architects like this part the best).
The role of architect, as a formalized job description, didn’t exist when I first started programming in a structured environment (as opposed to hacking before I had my first job). Over the years as software has gotten more complex the methodology has gotten more and more involved, thus taking a lot of upfront time before a single line of code is written. This is the so-called “waterfall” approach, i.e. think everything through, in detail, write it all up in specs, then start implementing. But, of course, this is slow. It’s especially slow for me in personal projects. So the world, esp. with the different mentality of development for the web (often called the “perpetual beta”) needed a different approach than the slow tedious product (or even IT) software development.
And then there was outsourcing, often to programmers who speak little of the human language of the architecture and/or product manager. So while the web development pushed for “agile” development, even the somewhat ridiculous “extreme programming” methodology (what I’m doing now and what I used to do decades ago before, painfully, learning the development process) there is still tension between various (and often highly dogmatic) “agile” methods, and often many problems when the development effort is multinational.
So the software world debates all this. Academics, who rarely have real experience with large scale software development, pronounce all sorts of “musts”, even down to the way to hold meetings. Meanwhile the people responsible are typically older and more senior, which means they predate much of this methodology and so, a bit, have to force themselves to use it. Sometimes those of us old school types laugh a bit (or a lot) at some of the silliness of these rigid methodologies.
Anyway out of all this the role of architect became more defined, to the point of becoming an explicit job description. I used to define it this way – programmers (done in the mud) develop individual classes or sometimes just parts of complex classes; “designers” (up a 1000 feet in the air looking down at the mud) come up with all the specs for a complete set of classes to build a “server”; “architect” (way up in the air, 10,000 foot level) puts together an entire system composed of many “servers” (until the whole concept of server got refined, architect and designer were pretty much the same job).
Well, I’ve done all these jobs. But now it’s just little old me. And I’m impatient. I want to see pixels on the screen, not slogging through detailed server and class definitions. In fact when I started this project I have no concept of how big it would get, big enough to in fact deserve an architecture, which I’m now building *after* getting individual bits to work, hence the big refactoring I’m having to do (not finished yet) in the shape generator. And the shape library, somewhat like a database (or data model), in other projects, has become central. But I didn’t realize that would happen so I neglected any real architecture specifications.
And, in doing this history, I neglected doing any “design” work and now I’m paying for it. Some people think charging into the code for quick results (the extreme programming model) is fine – the refactor as it becomes clear the code has gotten messy. Fine, BUT, refactoring is no fun, plus lack of design in the original code creates all sorts of bad practices. While I scoff a bit at design patterns there is some point to it all.
So, what does this all mean? Well, four days ago I had some classes (in some cases just stubs) to create this container, the compositeFile, that would be used for many parts of my ultimate project. I had built, without much design, except in my head, the basics to: a) create and initialize the container, b) close it after creation, and, c) open a (now) existing compositeFile. But I hadn’t actually tried to write any data in it. I knew approximately what I needed to do, but was tired of just writing pure code that didn’t do anything tangible, so I switched back to a top-down effort, actually rework the shape generator so it would need the compositeFile, before proceeding with the compositeFile development itself. This seemed reasonable and I thought would be quick. It wasn’t.
Now the compositeFile is more at the 1,000 foot design level than the 10,000 foot architecture level, but I messed up some important OOP considerations, most importantly encapsulation and abstraction, but also from the architectural POV, a very critical concept, “separation of concerns”. I now realize, in typical 20-20 hindsight that how the compositeFile is actually stored absolutely should be invisible to other classes. A compositeFile might just exist in memory, temporarily, or it might be a simple (albeit large) diskfile, or it might be based on a SQL server, or perhaps even something more abstract than that, out in the “cloud” (in more a metaphorically sense). This is the right way to do it. So how did I, somewhat mess that up, due to lack of upfront design and incremental development.
So let’s look at just one bit – blobs, or Binary Large OBjects. To a degree doing these right has plagued a lot of people. My last job was really building a new layer, under Microsoft SQL (really SharePoint, but it was the SQL behind SharePont that was our focus). SQL, somewhat like my little effort here started with focus on the standard relational database concept, i.e. tables, where the data was relatively simple types (numbers, dates, strings, etc). But what happens when you start getting big stuff, like images or videos or sound recordings). These don’t work in conventional databases. So really these blobs are handled differently, now crammed into tables, but shoved off in a different part of SQL with then a reference to them crammed in the table.
But blobs have a big problem with space management. I still laugh at some critics, decades ago, of using “paging” for virtual memory, instead of the more common at the time (both DEC and HP) of segments. I laugh because the criticism was that paging “fragmented” memory, when in fact it was segments that caused the actual problem with memory management in most early OS (like IBM’s MVT (or worse, MFT) before moving on to virtual memory). So blobs are exactly the same issue. If you try to store them on disk (or inside a database) and then blast away, deleting some, creating more, updating them (but their size is changes), pretty soon your storage is mostly wasted holes that are too small to save anything. “pages” (as in virtual memory, but also as now applied to file systems (it’s hard to believe it used to be done differently) break large chunks down into identical size chunks, which, of course, will now be scattered and discontiguous. But the space released by deleting one blob can now be reused for any new ones and in fact there is no “fragmentation” (i.e. some hole you can’t fit anything into).
This is just ancient history the kiddies ignore it, but even today some of these same things have to be dealt with. Sure, especially a couple of generations after I first saw it, SQL would do just what I need (and well), but that isn’t available to the amateur programmer. So I had to build a subset of it myself (or simple DBMs I can get do a lousy job of blob management).
So, in my (in my head) design I had a class ‘blob’ which would have some specialized subclasses based on the kind of data to put in the blob. OK, that’s simple enough. But the blob would have to be broken into ‘blocks’. And now just blobs, but all the internal structures in my compositeFile would be based on blocks, of which there would be numerous subclasses. Again, fine and dandy. But what is the exact relationship between the blob class group, the block class group, and the compositeFile clases – that’s were I fell down. A blob shouldn’t care how it’s stored because it’s just a bunch of blocks. But in reality blocks should also not care about how they’re stored. Both blobs and blocks can do their required function completely isolated from the storage. And that’s what I messed up, I sprinkled I/O code in these classes. Knowing I wanted some isolation, blocks can read and write themselves, but don’t know where – that’s outside (OK, that was appropriate, but I should have gone further). And since my compositeFile: a) may actually consist of multiple disk files, and, b) in fact not all blocks will be the same size, I put the concept of “seek” (i.e. turning a blocks unique id into a location on disk) in a higher level BUT not high enough. In essence I put some of the I/O in blocks and some in blobs and some in composite file, when in fact I should have put all I/O just in the composite file and the blob would have its methods to chase a chain of blocks and blocks would have its methods to read/write its data.
But my other mistake was “headers”. I initially, and over-simplistically defined a standard header for all blocks. But some blocks need more than the standard. So I began to kludge “extended” headers, in non encapsulated way, into each subclass of blocks. Now in fact I should have been much cleaner about this, reading/write a header should be independent of the data and, as well, reading/writing an extended header should be independent of both the standard header and the data. I commingled these two much. And while I’ve straightened it out some, in my new work, now I’ve got to go back and refactor the previous blocks. And in fact, now that I’ve evolved the design by trial-and-error I really need to refactor a lot. Now the concept of refactoring messy, but working, code is fine, but it’s no fun to actually do it, and often it will get forgotten under pressure to just keep moving (in my case I’m the impatient customer putting pressure on me, the developer, to get it done fast).
So that’s been my slog. That and lots of detailed coding messes of how you really build objects (C# is rather adamant I not do workarounds just in a hurry, but then, I have almost everything public (or at best protected) which is not good (I get really sick of type conversion or access violation compile errors, but instead of really fixing them, I kludge them to keep moving).
Now in a personal project, albeit it now a fairly large one, who cares about all this? So I create spaghetti code. This isn’t a product that will go through many generations and with a constantly changing staff (one reason for all the design methodology is the constant loss of the programmers that created one version and having to bring in new ones who have no clue what all this code mess is). But I do care, because I can imagine trying to either fix bugs or do small enhancements in weeks, or at most months, to the mess I just made. I’ll spend days trying to figure it all out just to make a tiny change (in fact, my experience is this still happens with most design and software these days, despite the methodology (and sometimes because of it)).
So where am I. Without any real stress testing it appears I can create and delete blobs, hurrah. But one big problem is a composite file is that any internal damage can cause a lot of data loss (in fact, maybe all). So something saved weeks ago gets lost during a crash saving something new today. I’ve experienced that before. So of course I added (as fun extra complexity) some redundancy (to confirm a blob is actually correct and intact) and also to recover a crashed file. That’s a good idea, but boy did it take a lot to implement. While all delete needs to do is find all the blocks in a blob and free them, that is probably less than 10% of all the code I wrote. The bulk of the code is various kinds of error checking and particular self-consistency checking (i.e. being skeptical that something else has trashed my blob, so don’t run off in the weeds while trying to delete or read it). Again, good idea, but really bogged me down.
So even though my methodology has slowed me down simultaneously I don’t really have as clean code as I need (even with no one giving me a grade on it, or complaining in reviews). If it’s such a mess I can’t easily keeping adding more functionality, without breaking everything else, I’ll end up being my own worst critic.
So, now, once again I’m optimistic – doing read will be a small extension to doing delete. We’ll see. With create, delete and read that’s enough to then integrate the compositeFile into the existing history stuff to actually persist shapes. Maybe tonight I’ll get that done if finally I manage to pick up the pace and I can report the milestone I expected to report several days ago.
mathcoloring 
I’ll be back
I’m still going to continue this project but have been distracted into something else that has totally adsorbed my attention, Project Euler. But I’m getting close to the end of what I can do there, so I’ll end my digression and return to this project, since all my readers are enjoying it so much.
Rate this: