It is currently Wed Nov 22, 2017 12:41 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 8 posts ] 
Author Message
 Post subject: Abstract Collaboration
PostPosted: Fri Feb 07, 2014 4:09 am 
Offline

Joined: Mon Mar 27, 2006 5:23 pm
Posts: 1339
Based on threads on iNES and SPC700 mnemonics ... I've been thinking a bit more abstractly about the nature of collaboration.

In almost all cases, there isn't one glorious truth. Everything has various trade-offs, and these trade-offs vary in importance based on the individual. If there were a universal correct way, I'd like to think we would gravitate toward it naturally. Much like in math, no one (sane) will argue the point that 2+2=4. That self-evident correctness is absent for a reason.

And so it seems that a necessary component to a successful project is to have a strong leader. When two equally talented individuals come up with separate solutions to the same problem, you need someone to pick one and move on to the next issue. In the corporate world, this is the CEO.

But we're not in a corporation, so we all work for free, with collaboration being our only tool. In the open source world, you see these roles filled by the likes of Linus Torvalds and Theo de Raadt. Yet here, these types of leaders tend to put up with a lot more. In the corporate role, you'd be fired for insubordination. In the open source world, you have to constantly deal with people undermining your efforts. I think this leads to extreme embitterment. Linus and especially Theo are extremely hated by many. They're extremely blunt. Yet they're effective at what they do. There's no arguing that Linux is the strongest open source operating system, and OpenBSD truly excels at security, making it a fantastic choice for a server.

I think that these projects are so successful not in spite of their leaders, but because of them. The brashness being an unfortunate yet necessary side-effect of dealing with the public constantly.

The real breakdown starts to occur when we approach the less organized communities, like emulator development. Every NES author here likely has their own ideas on how things should be done. And there's no clear leadership, nor am I suggesting there should be. Instead, everyone feels themselves to be an equal (or perhaps even above equal) participant in the community.

When I think of myself working on software, I continually come back to the fact I'm not being paid anything. Why would I listen to a bunch of other people who disagree with me, and produce something I can't personally respect? It's not so much a narcissism in thinking oneself is absolutely right. Certainly, input from others routinely shapes our opinions and is reflected in our work. It's just that they don't play a trump card over our own ideas completely. We still want our own expression.

And indeed we couldn't just go simply based on others' input: it's so diverse that you would doom yourself to an unholy mess that ended in failure. I've been down this road, too. You try and please everyone, and you end up with an emulator that supports 15 compression formats, ten different ways to load NES games, seventy five filters because one guy is just really passionate that 2xSaI-Bold is the one-true-filter. And all of this code interacts with each other, and you start getting into insane interdependencies and conflicts that bog you down to the point where 100% of your development time is spent trying to please everyone. And your UI ends up so convoluted that nobody can figure anything out anymore. They go to map their input and are presented with a 1200x800 window positively packed with 100+ controls because one of your users wants to play Super Mario Bros using his cell phone as the controller. And now your tablet user is complaining that the input config window won't fit on his 1024x768 screen. You just can't win when you try and please everyone.

Obviously I don't think anyone wants a world where every last piece of software has zero interoperability. I know I like all my desktop apps to look and function the same way. Code reuse in some cases is not just convenient, but essential. Think crypto libraries: not something you want to make up without decades of experience in the field and tons of peer review. Yet at the same time, I also shudder to think of a world with no choice.

This leads to an interesting philosophical conflict: when I use GNOME 3, and Windows Metro, I am absolutely aghast at how miserable the experience is. I couldn't imagine being forced to use one of these, with no Xfce alternative. I am constantly loudly vocal about how godawful the experience is to me. But I do recognize that some people, for reasons beyond my comprehension, enjoy these desktop environments. So how hypocritical am I when I write my own software to do things my own way, and force others using my software to change to suit my interests?

So ultimately, I think we're left in a situation where we have to strike a middle ground. Choice, but not too much choice. Restrictions, but not too many restrictions. A nightmare scenario for an idealist like myself.

And then I look to the motivations of individuals.

The first category I see are the conformists. Here, code reuse is quintessential. Surely one individual can't do better than the hundreds of individuals who have worked on the same problem already. There is no need to reinvent the wheel, you're just suffering from not invented here syndrome.

The second, are the idealists. Backward-compatibility is thrown to the wind. Seeing backward-compatibility as a crutch that infinitely bogs down progress, the optimal solutions to problems are sought. Nothing is sacred, everything is open to re-evaluation.

The third, are the arbitrators. Oft-futile efforts are made to bridge the gap between the first two categories. It's a great struggle, as sometimes this results in a stronger unified standard, and sometimes it results in a mediocre version of both competing standards. And of course, they have to deal with the extremes from both sides. Worse yet, their goal is to produce one standard, but they end up just producing yet another standard instead.

And rather than pigeon-hole people into one category, it's likely more of a scale:
<conformity .................. arbitration ................. idealism>

In particular, I feel the conformist category are very averse to change, the idealists are self-absorbed, and the arbitrators follow the road to hell paved with good intentions. Nobody gets off as the white knights here. Each has their strengths and weaknesses.

Absolute focus on reuse and compatibility would have us all using 16-bit DOS apps to this day. Absolute focus on idealism would have our entire ecosystem thrown out yearly with new languages and toolkits. And absolute compromise requires us to port our applications to five different toolkits using different languages for each. All of these options suck in their own way.

And that gives us the software scale of:
<compatibility and flexibility ............ compromise ............ elegance and simplicity>

The interesting paradox is that the category I personally reside in changes based on whether I am the developer or the user. When I am developing, I am a strong idealist. Yet when I am using others' work, I am more of a conformist. And I think the latter is mostly because I'm so disagreeable with some of the horrors out there. It's probably a reflection of me realizing how far from the mainstream my own ideas are, that I'd hate to be forced to follow this new trend that tablets and desktops should be exactly the same.

Hammering down exact definitions is quite challenging. An absolutely rigid conformity to a specific solution is a form of idealism to those who find that existing solution optimal. And an idealist ultimately wants everyone to use only one solution, despite the reality that the attitude to improve things ultimately produces the most options.

In closing ... no great epiphany or "this is what we should do" to this post. Just musings on the topic in an attempt to understand where all sides are coming from.


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 8:05 am 
Offline
User avatar

Joined: Sun Jan 22, 2012 12:03 pm
Posts: 5825
Location: Canada
Other aspects of open source can play into this train of thought as well, I think.

- The knowledge put into practice by a project is shared and available to everyone. Even if we don't like some parts of your software, we can still learn how you did the parts we like.

- Open source software can be thoroughly inspected, debugged, analyzed. Bug reports from skilled users sometimes come in with not just a good description of the bug, but a patch to fix it as well. The community develops a strong confidence in the software because they know exactly what's in it.

- An abandoned project can be revived and continued by someone else who is now interested in it.

- I can add my own customizations for personal use that the you wouldn't tolerate in the main branch. I can even share a patch for this with others who have enough skill to recompile the project for themselves.

- You might create a competing fork of my project that becomes comparably popular to my own. This is both good and bad. Obviously there's more choice here, but it cultivates confusion as well, especially as incompatibility grows between the forks. Nobody knows which is the "better" version (probably because there isn't one, objectively). People submit bug reports to me about your version. Complaints about features implemented in only one version irritate both of us. On the other hand, sometimes one of us will make the effort to integrate good changes from the other fork, improving the software for everybody. Other times, neither of us can be bothered to make the effort to do so.


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 9:06 am 
Offline
NESICIDE developer
User avatar

Joined: Mon Oct 13, 2008 7:55 pm
Posts: 1049
Location: Minneapolis, MN
All good points. The main problem as I see it with open source projects is that most have no chance of becoming an established, mainstream, newsworthy effort. The one with the idea ends up being the only one with any skin in the game. Even if your idea is genius...it's still just your idea. There's plenty of other geniuses out there with their own genius ideas, all with the same thought: "If I can just convince enough people that this is a great idea..." We are not a people motivated to collaborate. I can't tell you how many "code reviews" I've sat in that, despite the "leave your ego at the door" urgings, someone ends up getting defensive over a question.

In 2003, without a single bit of knowledge of the established NESdev community, I had the grandiose idea that I might be able to transform the world of NES homebrew development by developing an IDE that appealed to all users, novice and expert. I joined this forum and "came out" [so to speak] with my IDE in 2008. A full five years after I'd been toying privately with the idea of making something for others to use. My first post introduced my IDE. I was *startled* by the responses. One in particular: "I want to have your man babies." still makes me smile! I got a few that led me to perceive I'd actually made a difference and people would adhere to my ideals and come and help me achieve my goals. That happened...but not obviously to the same level as Torvalds. I'm no Torvalds. I can be an asshole...and I'm sure there's people at my workplace that despise me providing input to situations because I can be very blunt about how stupid some policies/procedures really are.

I've experienced plenty of what you described while developing my IDE publicly. Users that flit by with requests for additional features only to disappear once I've put time into developing the feature. Promises of lengthy collaboration from individuals with promising initial signs of it only to have the person up and completely disappear a week, month, year later. Not that I can complain...free help is free. I've had extremely amazing help from people next door [figuratively speaking] and people several thousand miles away. But in the end...they have lives...I have a life...and our goals, while coalesced for a moment on the singularity of my initial intent, inevitably diverge.

Ideas don't just become mainstream because you have them. Ideas require constant advertisement, constant integration with the environment in which they were hatched, constant upkeep. My IDE has been in existence longer than all of my kids...it is a fourth child, to me. My initial post introduced my initial attempt at an IDE, written entirely in MFC and thus fettered to Windows, and barely useable in any Windows virtualization elsewhere. I'd completely ignored the community that might not be using Windows native. Fast forward to 2010 and at the urging of a few of those extremely helpful individuals I mentioned, I'm now in Qt and developing a cross-platform IDE. So you might say I've also experienced the whole "throw out the baby with the bathwater" thing, writing my IDE basically a second time from scratch or nearly so. I "conformed" to the open-source, cross-platform ideology.

The idea I had of creating an IDE is still there and still strong. The actual implementation of the idea has morphed over the years and even gone through a complete change of clothing. I'd say it's 50% my own ideas of what the IDE should be, 20% my implementation of ideas from others, and 30% possibly-buggy dead ends that I [or one of my aforementioned expert help] started on but never quite finished.

I have been reading "Dreaming in Code" which describes the open-source efforts of OSAF in creating "a whole new PIM" called Chandler. Better than MS Outlook, better than any other PIM. That company had a CEO...and yet, 85% of the book was devoted to the sloughing-through of design decisions that seemed to fester for *ages* until someone woke up and realized they wouldn't get anything done if someone didn't make a decision. And make it FAST because the release was supposed to be done *yesterday*. The problem, as I saw it, was the CEO was just like any of us. He had his brainchild idea that, come hell or high water, was going to be realized. Compromise be damned. I downloaded Chandler and was appalled. Useless POS. But perhaps that was just because it was not like Outlook which I am familiar with. Did I feel compelled to collaborate with them and "make it better"? Hell no...too much shit to do to propel my *own* ideas forward even at a snails pace. I am nowhere near the sphere of people interested in developing PIMs, sure...so that might have biased me away from helping them. But it also didn't bias me toward wanting to create my own to "show them how its done". So in this case I guess I become a conformist...using Outlook because I know it. We're all familiar with the plethora of NESdev tools, each of which created by someone with a genius idea, and each of which with its own cadre of devotees that will scurry to the next tool only when it proves it's much much better/easier/faster than what they're currently using.

I live in all three of the modes you describe. I'm an idealist when it comes to pushing my own ideas and trying to convince others that my idea is worth a shit. "I'm writing a NES IDE...you should use it." I'm a conformist when I'm told that someone already did what I'm doing and it's better and I should use it. "Don't write your own C=64 emulator, VICE is *the shit*...you should integrate it with your IDE." I'm an arbitrator when I see someone else's work that I think could be made even better by enhancing it. "This FamiTracker community seems to be clamoring for a cross-platform version...let me try to build it."

You have to be ready to be in any mode at any given time. Insistence on being in any one all of the time dooms you to failure.

Thanks for the post...no personal attack meant, but this is by far the best of your posts I've read. The others are too geniusy-technical and long-winded for me, not having a lick [yet] of SNES knowledge. And...frankly, TL;DR is usually my gut reaction. You seem very knowledgeable and have a lot to say and contribute. Oh and one final question...why do you avoid the quote BBCode? 8-)


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 11:11 am 
Offline

Joined: Mon Mar 27, 2006 5:23 pm
Posts: 1339
> Obviously there's more choice here, but it cultivates confusion as well, especially as incompatibility grows between the forks. Nobody knows which is the "better" version (probably because there isn't one, objectively)

Exactly. For some things, this doesn't matter. Really doesn't matter if I'm the only user of my cross assembler or terminal-based calculator. But if you're trying to create an emulator, you need users to play games with it to spot bugs for you to fix. The libraries are too vast for one person to play through the entire thing each time they fix a bug. Automated testing can only get you so far: there's no magic "100% behavior validation test ROM" out there, sadly. So when you have 6000 NES emulators out there, it gets to be an issue. Similar problems arise with other fields, like document interchange formats. Or, most significantly to us, NES distribution formats. There are times when more alternatives directly results in more work necessary for someone seeking broad compatibility. So you get to play dictator and pick one, or pleaser and waste years of your life on interoperability, making an unstable mess of an application.

...

Probably wasn't fair to hone in on open source. There are certainly closed source emulators and tools and such around these parts. I was meaning more that the hobbyist circles like this, open or closed, tend to be very disparate. Getting everyone on the same page feels exactly like herding cats.

My views on open vs closed source are a lot more hostile. When you're in the business for money, it's vastly harder to make a profit when you give away your secrets. But in the freeware world, it just strikes me as inherently selfish. I think people have trust issues, and are more concerned about what they'll get than what they can give. I find the more I care about a field, the more closed source in it offends me. Yet I really couldn't care less that my nVidia graphics driver is closed source. By trust issues, I mean you have to be okay with knowing you helped make the SuperGnes author a few million dollars and didn't get a penny for it. If that sort of thing would eat at you, then you'd naturally be a lot more averse to releasing your source code. Closed vs open source has easy parallels to captalism vs socialism, but I don't think anyone wants a politics discussion here :P

When it comes to open source and forking / enhancing, meh. When a project gets too big, a fork can only happen when the owner goes nuts (Oracle being a prime example) or the community goes nuts (XFree86 license change being a prime example.) When it comes to smaller projects, people just write their own replacement. In the middle ground, I don't often see much forking outside of continuing dead projects. I've been trying to get someone to maintain a version of my emulator with more user-friendly features. Any time someone even starts to do so, they end up turning it into a big multi-emulator package instead. Which is fine, it just leaves me without having a user-focused UI to point people toward.

What I'd really like to see more of are parallel enhancement forks. Not "fork the codebase, never work with the original again", but more like, "keep a set of enhancements and apply them to each upstream release." I'm sure it happens, but less often than I'd like.

> Even if your idea is genius...it's still just your idea.

And indeed it's not even that your idea isn't spectacular. It's that other, different ideas are also spectacular. And your ideas have some flaws that you don't care about, and other ideas have flaws you do care about. It's one of the hardest things for me as a developer to make my own software less usable and enjoyable to myself, just to make others happy. I certainly would do things completely differently if I were selling software, of course.

> Ideas require constant advertisement, constant integration with the environment in which they were hatched, constant upkeep.

That they do. If any of us had the secret for how to ensure an idea caught on globally, we'd be billionaires instead of obscure hobbyist coders.

Every time I've tried to promote my ideas, it's just backfired entirely. You have this idea, and you want to tell people why they should use your software instead of some other competing software. So you try and take the high road: promote your benefits without disparaging your competitor. But no matter how nice you are, the second you say, "oh well I can also do this", you get the inevitable, "eh well so can the competitor." So you show a demonstrable example, and now you may as well have just insulted the competitor's mother.

Then you go and take the software you've spent five years making, and post it in a programmer congregating area like r/programming or YC, and your post is immediately flagged as advertising spam.

Then some guy spends 20 minutes making Flappy Bird and is now a millionaire :P

I've just given up on the idea of breakout success. It's not worth the trouble. I just make stuff I want to use, so it doesn't matter if nobody else uses it. It was fun to make, and it's even more fun to use.

> I downloaded Chandler and was appalled.

And it could be a great idea, if you take the time to relearn how to use it. Just like Gnome 3 or Windows Metro or anything else. How much time do you have to use something before you can say, "yeah this idea is awful"? It's rare to get someone who will give a new idea more than a minute of their time. I've tried my best, using the new OSes for a few weeks. All I could bear.

Improvements that change things don't just have to be a bit better. They have to be fantastically better for people to switch. And you have to make them use the idea for a while. And this absolutely infuriates people to no end. Keeping in mind that different people have different priorities. Eg game folders may be a big deal to me, but are an irrelevant detail to most, so change there is so much more difficult.

And yeah, sometimes the new idea really is just a steaming pile of shit. I just can't see how Gnome makes anything better. I'm still looking for the mythical tablets in stores that run Gnome 3, hoping maybe it'll be more understandable there what they are going for.

> I live in all three of the modes you describe.

Exactly, this is what is bugging me. I know a lot of people hate categorizing themselves, but I'd like to form some sort of chart (be it 1D, 2D, 3D, or more) to reflect my views on things, and use it as a relatability tool. Like those political compass charts. It's a nice quick summary of my thoughts on politics. Similarly easy to rank myself theologically with the 2D (a)gnosticism/(a)theistic chart. But when it comes to software ... it seems like I'm all over the map and it changes based on how much I care about something, whether I am working on it or someone else is, etc.

> The others are too geniusy-technical and long-winded for me; And...frankly, TL;DR is usually my gut reaction.

Well thank you, but ... these two posts weren't ridiculously long-winded for you? o.O

Wasn't sure anyone was going to read it. Just helps me to type out my thoughts sometimes.

> Oh and one final question...why do you avoid the quote BBCode?

a) takes a lot longer to write {quote}...{quote} than > ... [laziness]
b) how I've always done it with e-mail for years before bbcode caught on [habit]
c) uniqueness? :P


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 11:50 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19239
Location: NE Indiana, USA (NTSC)
byuu wrote:
Then you go and take the software you've spent five years making, and post it in a programmer congregating area like r/programming or YC, and your post is immediately flagged as advertising spam.

I think that's why some people write software to please other people, so that they can name-drop it in an on-topic context.

Quote:
I'm still looking for the mythical tablets in stores that run Gnome 3, hoping maybe it'll be more understandable there what they are going for.

Look for any x86 tablet, such as the ASUS Transformer Book, and install a dual-boot Linux distribution including GNOME 3 on it.


Top
 Profile  
 
PostPosted: Fri Feb 07, 2014 5:48 pm 
Offline
User avatar

Joined: Mon Sep 27, 2004 8:33 am
Posts: 3715
Location: Central Texas, USA
I have no desire to lead. If people can't decide on something, oh well. In discussions my goal is simply clarity of everything and honest about goals and tradeoffs. With a clear picture, people can understand each others' wants (they need no justification; if you want something due to aesthetics, fine, no shame or dismissal just because of that). Solutions can be explored which best meet everyone's wants, assuming there are no tradeoffs. Where they are, if people have good will then they will naturally make small sacrifices where it benefits another party significantly.

I see a huge source of problem in communities like this of others acting like they know why people want things and dismissing based on this characterization. It leads to responding not to their technical arguments, to what the person is saying, but to what you think you see as the underlying cause or intent. Then you respond in a way that doesn't address what they were saying, instead tries to pressure them into doing things how you think they should be done. All these labels of people just get in the way of perceiving the present and what's happening, hearing what people are saying. So often I have people assuming my questioning is going in some direction that triggers their "oh know, this is happening, and I know this approach won't work, so I have to stop it now" warning, and they refuse to answer the questions and get more and more accusative that I'm doing something that I'm not. I see all this kind of thing as a weakness, a lack of knowing how to be assertive of oneself so having to make pre-emptive attacks to stop people from doing things you'd be powerless to respond to.


Top
 Profile  
 
PostPosted: Sat Feb 08, 2014 2:08 am 
Offline

Joined: Mon Mar 27, 2006 5:23 pm
Posts: 1339
> I have no desire to lead.

I definitely do. I don't have any authority to do so, as we are all equal peers, but I at least make decisive actions with regards to my own software. It really seems to me that any successful project needs a strong leader to avoid bike-shedding and design-by-committtee.

> Where they are, if people have good will then they will naturally make small sacrifices where it benefits another party significantly.

Where do you draw the line when it comes to doing what you want to do, and what others want you to do? The more you do the latter, the less fun the entire project becomes.

I like to think I make smaller sacrifices, like maintaining and supporting importing games in compressed ZIP archives with iNES headers, incorrect extensions, using heuristics and such. But on the bigger issues, they're what define what I am trying to accomplish.

> I see a huge source of problem in communities like this of others acting like they know why people want things and dismissing based on this characterization.

I think people naturally want to relate to other parties and will try and understand their intentions.

Intentions are really, really important to understand. Does it make sense for a socialist liberal and an authoritarian conservative to discuss how best to help the poor? Or an agnostic atheist to discuss the origins of life with an evangelical Christian? I know you're going to say that it's important for people to hear opposing viewpoints, but eventually the viewpoints are so far apart that you'll just be talking through each other instead of to each other. It's a complete and utter waste of time like that Bill Nye v Ken Ham debate.

So when it comes to discussing a controversial idea, you want to make sure you're talking to someone who has the potential to change their mind. If the distance between you and them is too vast, it's not worth trying in my opinion.

I definitely get where it becomes really annoying when others incorrectly categorize you, or worse, put words in your mouth. But when done right, and you correctly cut through to the root of the problem, you can save a lot of time in discussion. There's a reason doctors and psychologists generally focus on the cause instead of just on the symptoms.

What I think would really help here would be if people were more honest and upfront about the areas that they won't compromise on. You did that in your threads, and I overlooked them, which was my fault. But it certainly isn't common.

> It leads to responding not to their technical arguments, to what the person is saying, but to what you think you see as the underlying cause or intent.

No argument there. It has always been an extreme challenge for me to keep everyone on a technical level. People bring far too much emotion into things. Ad hominens being the lowest form of response.

> instead tries to pressure them into doing things how you think they should be done.

Everyone does that. That's the whole point of having a discussion.

I want to convince you that I am right, you want to convince me that you are right.

The absolutely critical detail is that both parties have to be willing to give at least a little. Otherwise the entire debate is disingenuous.

But if you go into an argument with an attitude of everything being shades of gray, and lack any assertion of your beliefs, you'll never convince anyone of anything. Nor will you even get so much as a strong critique against your idea if you abandon it right away. How can you expect people to take you seriously if you're wholly pliable?


Top
 Profile  
 
PostPosted: Sat Feb 08, 2014 8:02 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19239
Location: NE Indiana, USA (NTSC)
blargg wrote:
I see a huge source of problem in communities like this of others acting like they know why people want things and dismissing based on this characterization. It leads to responding not to their technical arguments, to what the person is saying, but to what you think you see as the underlying cause or intent. Then you respond in a way that doesn't address what they were saying, instead tries to pressure them into doing things how you think they should be done.

I wonder how much of the "why would you want that?" tendency comes from years of having to deal with people who fail to "describe the goal, not the step", as ESR put it, leading to an XY problem. When one thinks a step is being misapplied, one might try to discern the underlying goal.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 10 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group