########################################################################## ## ## # # # ### # # # # #### # #### #### ##### #### # # # # # # # # # # # # # # # # # # # # # ###### # # # # # # # ###### # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ## ## ##### ##### #### #### # # # ##### # ## ## ########################################################################## +==============================================================================+ | Title: [ About ] /------------------------------------------------------------------------------\ Welcome to my web log. I will try to update this frequently with my thoughts and such. I won't, of course, but I will try to. If you have any questions or comments, please email them to ????????????????????????, and I will post them on here when I get a chance. This page is Valid HTML 4.01 Strict and contains Valid CSS 2.1. Hopefully it will render as intended in most browsers on most operating systems. Ironically, it doesn't render correctly on console-based browsers. To view this page from a UNIXish operating system, run the below command: wget -qO - http://joshodom.net/weblog/plain.txt | ${PAGER:-more} \------------------------------------------------------------------------------/ +==============================================================================+ | Title: [ Open-Source OS Design Pt. 2 ] | Or: [ Concerning File System Structure and Resource Locations ] | Date: [ Monday, 11/28/11 at 11:41 PM | 1322545299 ] /------------------------------------------------------------------------------\ One good idea they had when designing Unix was to put all resources of one type in one location, so you only have to look in one place to find what you are looking for. For example, you can find system binaries in /bin/, you can find user or application binaries in /usr/bin/, you can find user libraries in /usr/lib/, and so on. Well, this is a good idea on systems which only have a handful of applications, but on desktop systems which have thousands of applications and libraries and other such resources, it becomes impossible to manage. Thus, package managers were born, which keep track of library dependencies between applications, and keep track of all of the installed files per application and per library, which makes it possible to uninstall programs, after you've installed them. However, I see this as a kludge or a hack -- a way to get around the fact that Unix systems aren't inherently maintainable. Windows' solution to this is pretty reasonable, but doesn't solve everything. Placing each application in its own directory (in Progra~1) separates out the main essentials, per application, but system libraries are still distributed in the Windows\System32 folder (and thereabouts). I don't fully understand how OS X does it, but I know it uses .app folders, which seem like a good idea to me. I have a few ideas, but I am not completely sure how to implement them. First, core system utilities should be installed using the standard Unix paradigm. Each application which is not core, should be placed in its own sub-folder, and that sub-folder will mimic the standard /bin/, /share/, ... layout of a Unix filesystem. The package heirarchy should allow for structuring, so all applications which are under a certain project (e.g. gnome) can be grouped together. Then, there would need to be some application or some mechanism built into the kernel which would squash down the necessary applications, libraries, down into the base filesystem, so that the applications can run as intended. Also, some kind of dynamic reverse-squashing would need to happen, in case, say, you want to look at all system logs. With the package separation, all log files would reside in their own package folder, so there would need to be some way to dynamically get all of the log/ folders from the applicable applications, and squash them into one folder. However, this shouldn't be too critical, because it's easy enough to find the applicable log file, when you know which application is creating the log file. There would need to be some versioning, so multiple versions of an application can be installed side-by-side, with only one "primary" version of the application (the primary version being the one found on the path). There would also need to be at least some very basic registry-like functionality, so libraries know who they are being used by. This would have to be implemented per package, I believe, so I am not completely sure that this is possible without creating yet another package manager. In summary, I think that fixing package management from the bottom-up instead of from the top-down, has the potential to give desktop users more control over their applications and fewer difficulties in installing and maintaining their applications. \------------------------------------------------------------------------------/ +==============================================================================+ | Title: [ A Heart ] | Or: [ The ubiquitous Western symbol for love ] | Date: [ Wednesday, 11/23/11 at 6:54 PM | 1322096073 ] /------------------------------------------------------------------------------\ ??? ??? ????????? ????????? ??????? ????? ??? ? \------------------------------------------------------------------------------/ +==============================================================================+ | Title: [ Open-Source OS Design Pt. 1 ] | Or: [ Concerning Kernels and Device Drivers ] | Date: [ Wednesday, 11/23/11 at 5:26 PM | 1322090806 ] /------------------------------------------------------------------------------\ Word of warning: this article is based primarily on speculation, and what I think good direction would be. However, I have never written a device driver, and I have never created a hardware device that interfaces with a computer. I can only talk about my frustrations, and the direction I think is most sane. Linux and FreeBSD and the other free Unixes have always had the problem of hardware support, as far as I can remember (although it's gotten significantly better over time). The problem is multifaceted, I believe -- 1) Almost all consumer hardware is designed to be run on a windows platform, so the linux support for this hardware is either non-existent, community-supported (by people who may not really understand the hardware or who may not understand driver design), or given very low priority by the company (in the best case). 2) Even if the company who makes the hardware does support linux, since the linux kernel is under constant revision, it may no longer work with next year's kernel. 3) Even if your driver is supported by next year's kernel, if you decide to upgrade to it, you will have to re-install the custom driver, and probably manually. 4) Linux, FreeBSD, OpenBSD, NetBSD, and a handful of other free operating systems use a different structure for their kernel. What this means is that a driver written for one will need to be re-written for each of the others. 5) Drivers are almost always written in C. C is a language with its merits (simplicity and cross-platform support, to say the least), but it has some fatal shortcomings. Although well-structured C code can be readable and efficient, it's increasingly more challenging to make well-structured C code as the project increases in size. Writing things in C can be extremely tedius. Finally, most C code I've seen in the wild is structured in a way that makes no intuitive sense (to me, at least). One exception to this is the OS X (XNU) kernel, which uses C++ and object-oriented principles for its drivers. Although this (theoretically) makes writing drivers much easier, it is also specific to drivers written for the XNU kernel. My proposed solutions to these problems: sorry Richard Stallman, I am going to make you cry, but drivers from the hardware manufacturer must be distributable in binary form, and this binary must work the same across operating systems. An example of where this works, is with the windows NDIS interface, and linux leveraging that (ABI) interface by providing NDISwrapper, which allows windows network drives to run natively on a linux operating system. I think of device drivers, not as software, but as extentions to the device's hardware. Basically, a wrapper, so the device can use whatever hardware interface it wants, but still has a uniform interface to the operating system. Some device manufacturers do offload a lot to their drivers, but I still think that drivers should be treated as extensions to the hardware. I don't know how to enforce standard ABIs across operating systems (maybe just try to implement windows' main ABIs?), but I think that would be a significant step in the right direction. However, I don't know of a good way to make this platform-independent, without compiling the drivers down to an intermediate form (e.g. CIL, LLVM IF, or Java bytecode). This might be a step in the right direction, but I don't know how one would go about designing an ABI which is architecture-independent. Strong ABI interfacing would mostly resolve issues 1, 2, and 4. However, linux distributions (at the very least; I would imagine you have the same issues with FreeBSD and the like) use an entirely new set of drivers when you upgrade the kernel. Now, NDISwrapper might be a little bit different; it could be that it keeps all of its drivers in a central repository, which doesn't change between kernel versions. Let's say you're a casual user, and you decide to do a system upgrade. You shouldn't need to know anything about your device drivers, and which ones are compatible with your new kernel, or anything about recompiling custom drivers for your upgraded kernel. The ways to do this: first off, the ABI interface layer (akin to NDISwrapper) would need to persist between kernel upgrades. Secondly, there should be some versioning support so it's possible to install a new version of a driver, and revert back to the old version if there's some serious compatibility issue. This support would have to be specific to the kernel at hand (i.e. Linux and FreeBSD would have to present their own solutions). I don't know enough specifics to give advice on how this should be done. Finally, problem 5. Problem 5 is really a non-problem. This is for a few reasons. First, thinking of the driver as an extension of the device's hardware, it should be as lean and efficient as possible. Often times, this efficiency can only come from a very low-level language. Secondly, the device driver (in principle) should be a very small program, implementing only minimal functionality (to translate requests between OS and hardware). Finally, there's nothing preventing programmers from writing a driver in a high-level language (as long as that language doesn't have any runtime components), and compiling it down to binary. In fact, it would be possible to even use a language which does have runtime components (e.g. automatic garbage collection), as long as the runtime components are also built into the driver's binary code. So in a nutshell, I think that strong ABI interface standards, strong kernel and driver versioning (in open-source kernels), and (less-critically) support for drivers compiled to an intermediate binary form will resolve a lot of the issues which desktop linux (or freebsd or...) users face. P.S. Richard Stallman, you don't have any place to criticize kernel design, because your kernel is 20 years old, and still doesn't work. \------------------------------------------------------------------------------/ +==============================================================================+ | Title: [ About Open-Source OS Design ] | Or: [ Why I think open-source software needs new direction ] | Date: [ Thursday, 11/17/11 at 5:10 PM | 1321571436 ] /------------------------------------------------------------------------------\ So I've been doing a bit of research, and a bit of thinking, and I am not convinced there's any reason for me to make my own OS kernel. There seem to be a plethora of kernels already out there that are hobbyist in nature, with many different design strategies, so I would be basically re-doing what has already been done, albeit suboptimally. So, although I think it would be fun to write a program and load that program in ring 0 (i.e. running as the OS kernel), I don't think I'll pursue it any further than that, unless I have very strong direction which isn't already explored. What I'd like to do instead is critique some of the directions which are currently being followed by the large open-source operating systems. Specifically, I want to critique linux and all of the projects surrounding and involved with it. First off, I'd like to say that what linux (and the various BSDs) does, it does really well. It has amazing cross-platform support, support for a wide range of hardware devices (which is absolutely critical for an operating system which seeks wide adoption), and the entire system is fairly open by design. My critiques, then, will be of the shortcomings of various components which make up the operating system as a whole, especially in regard to widespread adoption as a desktop operating system (currently, it has widespread adoption for servers and embedded systems). A lot of the current strategy seems to be fixing problems from the top down (i.e. patching areas where there's lacking or broken functionality, instead of fixing the architecture which has the problem in the first place). There's good reason for this -- when changing fundamental structure, you will inevitably break backwards-compatability of a lot of existing applications. Also, if you make a change from the top down, you don't have to worry about standards or widespread support, whereas a change from the bottom up must be made as some kind of standard, to be useful in any way. I will be giving critiques of each module/subsystem in individual blog posts, along with my suggestions of better ways to do it, and what I consider to be good direction/vision. Stay tuned! \------------------------------------------------------------------------------/ +==============================================================================+ | Title: [ On Operating System Kernels ] | Or: [ How I'd like to make my own ] | Date: [ Tuesday, 11/15/11 at 8:54 PM | 1321412077 ] /------------------------------------------------------------------------------\ I've been a bit intrigued about OS Kernels my entire life, it seems (or ever since I've known what they were). When I was a tiny kid, it was my dream to make an operating system of my own. Since then, I've learned a lot about operating systems in general, I've messed around with linux (including building a minimal linux-from-scratch environment a couple times), but I've never really "made my own" operating system. The key aspect to making one's own operating system would be to build your own kernel, I think. The kernel provides a basic abstraction layer that allows processes to run simultaneously and to access hardware in a simple manner. The kernel's structure will define how drivers are written for it (i.e. how the kernel can talk to new hardware), and how programs can interact with the kernel (i.e. the available system calls). In order to "make my own" operating system, I will simply need to make my own kernel, and get it to do something. I have low expectations that I will ever get it to do anything useful, but apparently making a trivial kernel is straightforward. I guess my first step will be to make a working kernel, and once I've completed that, I'll try to make it do something interesting. I'll post any updates on here. \------------------------------------------------------------------------------/ +==============================================================================+ | Title: [ Physics is Interesting ] | Or: [ Gravity and the tides ] | Date: [ Friday, 11/11/11 at 3:16 PM | 1321046213 ] /------------------------------------------------------------------------------\ My buddy James J. pointed out an interesting conundrum to me -- how is it that the miniscule effect of gravity could cause such a significant change in the tides? He pointed out that the moon's gravitational effect is some 10 millionth of the amount of the earth's gravitational effect on us. The few times I thought about this in my life, I could never figure out how such a small effect could cause such a big change. I remember reading some time back the explanation that the reason we have two tides (one on the side of the earth nearest the moon, and one farthest from the moon) is because not only is the moon's gravity pulling on the water, but it's also pulling the earth away from the water on the far side of the earth. That explanation never made sense to me, because I had figured the moon would be pulling the eart just about evenly, from all points, so there's no reason it would be pulling the earth away from the water any more than it would be pulling the water behind the earth. James then pointed out the solution. There are only two points on the earth where the earth's force of gravity is parallel to the moon's force of gravity. However, there's always a circle around the earth where the moon's force of gravity is at a 90 degree angle to the earth's force of gravity, which makes that tiny force have no opposition. Well, it does, after the moon's gravity pulls the water around the rim of the earth toward the moon. See this link for a good explanation (in terms of centrifugal force from the earth's rotation, but the same principle applies). [ www-istp.gsfc.nasa.gov/stargaze/Srotfram1.htm ] Now I have to wrap my head around the notion that angular momentum is a definite property of matter in the framework of general relativity, but linear momentum is not.... According to wikipedia, angular momentum is the 2-form Noether charge associated with rotational invariance. \------------------------------------------------------------------------------/ +==============================================================================+ | Title: [ Hello World ] | Or: [ How I made a new web log ] | Date: [ Friday, 11/11/11 at 3:53 AM | 1321005192 ] /------------------------------------------------------------------------------\ I am testing out my new web log. I want to see how it looks. I made a little python snippet that turns a string into a sequence of HTML character codes. This is useful, for example, if you want to obscure your email address so spambots can't see it. See this page's source for an example. ## htmlify('Abc') => 'Abc' def htmlify(text): nums = map(ord, text) strs = [] for x in nums: strs.append("&#%d;" % x) return "".join(strs) \------------------------------------------------------------------------------/ ########################################################################## ## ## # ## ## ##### ##### #### #### # # # ##### # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ###### # # # # # # # ###### # # # # # # # # # # # # # # # # # # # # #### # #### #### ##### #### # # # # ### # ## ## ##########################################################################