title: How can I estimate the entropy of a password?

text: Having read various resources about password strength I'm trying to create an algorithm that will provide a rough estimation of how much entropy a password has. I'm trying to create an algorithm that's as comprehensive as possible. At this point I only have pseudocode, but the algorithm covers the following:   * password length   * repeated characters   * patterns (logical)   * different character spaces (LC, UC, Numeric, Special, Extended)   * dictionary attacks It does NOT cover the following, and SHOULD cover it WELL (though not perfectly):   * ordering (passwords can be strictly ordered by output of this algorithm)   * patterns (spatial) Can anyone provide some insight on what this algorithm might be weak to? Specifically, can anyone think of situations where feeding a password to the algorithm would **OVERESTIMATE** its strength? Underestimations are less of an issue. # The algorithm:               // the password to test     password = ?     length = length(password)          // unique character counts from password (duplicates discarded)     uqlca = number of unique lowercase alphabetic characters in password     uquca = number of uppercase alphabetic characters     uqd   = number of unique digits     uqsp  = number of unique special characters (anything with a key on the keyboard)     uqxc  = number of unique special special characters (alt codes, extended-ascii stuff)          // algorithm parameters, total sizes of alphabet spaces     Nlca = total possible number of lowercase letters (26)     Nuca = total uppercase letters (26)     Nd   = total digits (10)     Nsp  = total special characters (32 or something)     Nxc  = total extended ascii characters that dont fit into other categorys (idk, 50?)          // algorithm parameters, pw strength growth rates as percentages (per character)     flca = entropy growth factor for lowercase letters (.25 is probably a good value)     fuca = EGF for uppercase letters (.4 is probably good)     fd   = EGF for digits (.4 is probably good)     fsp  = EGF for special chars (.5 is probably good)     fxc  = EGF for extended ascii chars (.75 is probably good)          // repetition factors.  few unique letters == low factor, many unique == high     rflca = (1 - (1 - flca) ^ uqlca)     rfuca = (1 - (1 - fuca) ^ uquca)     rfd   = (1 - (1 - fd  ) ^ uqd  )     rfsp  = (1 - (1 - fsp ) ^ uqsp )     rfxc  = (1 - (1 - fxc ) ^ uqxc )          // digit strengths     strength =     ( rflca * Nlca +        rfuca * Nuca +       rfd   * Nd   +       rfsp  * Nsp  +       rfxc  * Nxc    ) ^ length          entropybits = log_base_2(strength)      A few inputs and their desired and actual entropy_bits outputs:               INPUT           DESIRED        ACTUAL     aaa             very pathetic  8.1     aaaaaaaaa       pathetic       24.7     abcdefghi       weak           31.2     H0ley$Mol3y_    strong         72.2     s^fU¬5ü;y34G<   wtf            88.9     [a^36]*         pathetic       97.2     [a^20]A[a^15]*  strong         146.8     xkcd1**         medium         79.3     xkcd2**         wtf            160.5          * these 2 passwords use shortened notation, where [a^N] expands to N a's.     ** xkcd1 = "Tr0ub4dor&3", xkcd2 = "correct horse battery staple"      The algorithm does realize (correctly) that increasing the alphabet size (even by one digit) vastly strengthens long passwords, as shown by the difference in entropy_bits for the 6th and 7th passwords, which both consist of 36 a's, but the second's 21st a is capitalized. However, they do not account for the fact that having a password of 36 a's is not a good idea, it's easily broken with a weak password cracker (and anyone who watches you type it will see it) and the algorithm doesn't reflect that. It does, however, reflect the fact that xkcd1 is a weak password compared to xkcd2, despite having greater complexity density (is this even a thing?). How can I improve this algorithm? # Addendum 1 Dictionary attacks and pattern based attacks seem to be the big thing, so I'll take a stab at addressing those. I could perform a comprehensive search through the password for words from a word list and replace words with tokens unique to the words they represent. Word-tokens would then be treated as characters and have their own weight system, and would add their own weights to the password. I'd need a few new algorithm parameters (I'll call them lw, Nw ~= 2^11, fw ~= .5, and rfw) and I'd factor the weight into the password as I would any of the other weights. This word search could be specially modified to match both lowercase and uppercase letters as well as common character substitutions, like that of E with 3. If I didn't add extra weight to such matched words, the algorithm would underestimate their strength by a bit or two per word, which is OK. Otherwise, a general rule would be, for each non-perfect character match, give the word a bonus bit. I could then perform simple pattern checks, such as searches for runs of repeated characters and derivative tests (take the difference between each character), which would identify patterns such as 'aaaaa' and '12345', and replace each detected pattern with a pattern token, unique to the pattern and length. The algorithmic parameters (specifically, entropy per pattern) could be generated on the fly based on the pattern. At this point, I'd take the length of the password. Each word token and pattern token would count as one character; each token would replace the characters they symbolically represented. I made up some sort of pattern notation, but it includes the pattern length l, the pattern order o, and the base element b. This information could be used to compute some arbitrary weight for each pattern. I'd do something better in actual code. # Modified Example:               Password:          1234kitty$$$$$herpderp     Tokenized:         1 2 3 4 k i t t y $ $ $ $ $ h e r p d e r p     Words Filtered:    1 2 3 4 @W5783 $ $ $ $ $ @W9001 @W9002     Patterns Filtered: @P[l=4,o=1,b='1'] @W5783 @P[l=5,o=0,b='$'] @W9001 @W9002          Breakdown:         3 small, unique words and 2 patterns     Entropy:           about 45 bits, as per modified algorithm          Password:          correcthorsebatterystaple     Tokenized:         c o r r e c t h o r s e b a t t e r y s t a p l e     Words Filtered:    @W6783 @W7923 @W1535 @W2285          Breakdown:         4 small, unique words and no patterns     Entropy:           43 bits, as per modified algorithm      The exact semantics of how entropy is calculated from patterns is up for discussion. I was thinking something like:               entropy(b) * l * (o + 1) // o will be either zero or one      The modified algorithm would find flaws with and reduce the strength of each password in the original table, with the exception of `s^fU¬5ü;y34G<`, which contains no words or patterns.


title: What php programmer should know?

text: I've dig the database here and didn't found any answer for my question. What is a standard for a php programmer to know? I mean, literally, what group of language functions, mechanisms, variables should person know to consider oneself a (good) php programmer? (I know 'being good' is beyond language syntax, still I'm considering syntax of plain php only) To give an example what I mean:   * functions to control http sessions, cookies   * functions to control connection with databases   * functions to control file handling   * functions to control xml etc.. I omit phrases like 'security' or 'patterns' or 'framework' intentionally as it applies to every programming language. Hope I made myself clear, any input appreciated :) **EDIT 1:** > **Note:** > > Michael J.V. is right claiming that databases are independent from language, > so to put my question more precisely and emphasise differences: > > Practises or security, are some ideas to implement (there is no 'Pattern' > object with 'Decorator()' method, is there?) while using databases means > knowing a mysqli and a set of its methods. **EDIT 2:** It all came to me after some interview task: > Count seconds, days, months, years elapsed since date input till now. So I had opened the php docs to check date functions. After I had done tasks the interviewer told me he wasn't satisfied with my actions since datetime() is something everyone should know. That's why I'm asking all this. What everybody should know..


title: New to programming, where do I start?

text: So ... I'm a sysadmin. There are a number of instances where scripting would come in handy(mostly user management ... creating groups, assigning permissions, deleting expired accounts, etc), and I'd like to make a fullblown program. But I don't even know where to start. **Edit** A lot of comments made sense along the lines of "learn what you want to learn." So ... I need to figure out what I want to learn.   A: Is there no such thing as a "beginner" or "building block" language? It wouldn't be any easier to learn C# and move into C++ than it would to just go straight into C++?   B: How do I choose what to learn? I'd like to write a game for my first real project, if that helps narrow it down. (aside from "Hello world" ;) ) There are so many languages out there I don't know where to begin. I don't even know the real difference between C#/C/C++.   C: Once I _have_ picked a language, what's the best way to start learning it?


title: Being a good mentor

text: If you had a student or junior developer who was competent but just had no real world experience how do you think it would be best to mentor them or what information could you give them that would ease their transition from newbieness into leetness?


title: In TDD, if I write a test case that passes without modifying production code, what does that mean?

text: These are Robert C. Martin's rules for TDD:   * You are not allowed to write any production code unless it is to make a failing unit test pass.    * You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.    * You are not allowed to write any more production code than is sufficient to pass the one failing unit test. When I write a test that seems worthwhile but passes without changing production code:   1. Does that mean I did something wrong?   2. Should I avoid writing such tests in the future if it can be helped?   3. Should I leave that test there or remove it? * * * **Note:** I was _trying_ to ask this question here: Can I start with a passing unit test? But I wasn't able to articulate the question well enough until now.


title: The importance of using Configuration Management tools in a Software project for controlling the changes

text: Can we control the changes on a special artifact while our configuration management tool doesn't work properly? Can we do it ourselves?


title: How to use MVC in practice

text: I'm very confused. I can't even begin to understand how MVC would be implemented outside of web development. This might seem like too general a question, but how would one apply MVC. I have the following general questions:   1. Are M, V, and C all meant to be one class each, or many. If many, how would that work.    2. Most classes I've made previously have had their data inside them, not in a separate class. How would this work with MVC? 1) For instance, lets say you have a class where you take care of a virtual dog. I would think that you would make a Dog class with, for example, a "bark" command that would play a sound along with a name variable and a coat_color variable. I know this is very simple, but how would this fit inside MVC? It seems that you would end up with MC and V, where the information (model) AND the controls were in the Dog class, which would maybe access swing, or whatever library, to update the view. 2) Or, what about a program like sims (simplified version) where each person would have their own information... would you put all that info in another class? Sorry if these are all based on giant misconceptions, but I'm pretty confused. Right now I'm using Java, if that matters for MVC...


title: Moment to release an app to the App Store

text: Is there a good moment to release an app? Is there a bad moment to release an app? For example, maybe Spring/Summer won't get as many sales as the Fall/ Winter. In the sense that kids getting out of school will be spending less time in front of the screen, more time hanging out outside, etc. Any opinions on the matter?


title: Big project layout : adding new feature on multiple sub-projects

text: I want to know how to manage a big project with many components with version control management system. In my current project there are 4 major parts.   1. Web   2. Server   3. Admin console   4. Platform.  The web and server part uses 2 libraries that I wrote. In total there are 5 git repositories and 1 mercurial repository. The project build script is in Platform repository. It automates the whole building process. The problem is when I add a new feature that affects multiple components I have to create branch for each of the affected repo. Implement the feature. Merge it back. My gut feeling is "something is wrong". So should I create a single repo and put all the components there? I think branching will be easier in that case. Or I just do what I am doing right now. In that case how do I solve this problem of creating branch on each repository?


title: Should I expect recent grads to be familiar with basic programming concepts?

text: It seems there's a HUGE discrepancy between what I expect someone who's studied programming for a few years at university, and what one actually knows. I don't feel like I'm asking overcomplicated questions in interviews. Some of my usual questions are :   * What's the difference between a reference type and a value type?  If it seems the interviewee doesn't really understand his own answer, or if he doesn't know the terminology I'm using, I get into more details by asking him to explain me what happens when I write int i=0; in a method, what about object o=0, object o = new MyClass(), etc... Basically, I do everything I can to trick the interviewee into telling me about the callstack, the heap etc, and I try to stay into language-agnostics concepts. If the interviewee tells me he did a lot of C, or C++, or c#, I dive deeper into the specific language, and possibly into the implementation details. If needed, I ask the interviewee what a callstack is, or where arguments passed to a function in the imperative language of his choice are stored. most of the interviewees have just no idea about what a callstack is, let alone boxing considerations, etc.   * What's the difference between an abstract class and an interface. In which cases should you use one over the other ?  Usually, I also ask them to imagine a design of a small library with a use case aiming at using some inheritance and some abstract factories Most of the interviewee have just no idea of what inheritance real purpose might be. They usually know some keywords (virtual, override, etc), but don't really know when to use them, let alone explaining what a virtual-table is. Even-though I screen CVs beforehand, even for people with 5 years of experience in real life projects involving complex architectures, I would say less than 25% of all my interviewees can answer those two questions properly. And when I say properly, I don't mean 'in-depth'... just to have an approximate idea of what the concept is. Regarding juniors, I'm fine with hiring someone who doesn't know how to organize his time very well, or someone who's not used to industrial build processes for example, but I have the feeling that if one hasn't heard the word "callstack" after a few years of studying Computer Science, he's either stupid, or unmotivated, or chose his university very unwisely. Do you think I'm too extremist here ? Is it common to learn these basic concepts after you've completed university ? Do you know of people who weren't familiar with these, and became very good software engineers after a few years ? And do you think my company might have an issue in attracting talented people, or do you experience the same kind of issues with in your own hiring process? * * * Edit. regarding the "immediate type" thing, it was just a literal translation from French to English, as we usually do our interviews in French. I've fixed it in my question. But still, I think you all understand perfectly what I meant, which kind of makes my point, doesn't it?


title: difference between passing a value vs passing a function as a parameter in scala

text: I often hear that scala has a ability to pass function as a parameter to another function. I would like to know the difference between passing a value as a parameter vs passing function itself as the parameter. How the latter approach brings advantage to the programming. If functions are passed as the parameter how the object references are being created since functions are not tied to any objects.


title: Aspergers Programmer and Learning Difficulties. Best way to overcome them?

text: I have Aspergers Syndrome, and a effect of Aspergers for me is that I can find it difficult to understand concepts from reading material unless I have some idea or mental visualization about what it is I'm reading - which is often not possible because most the time we read something because we don't know about it in the first place. But, I have to be able to grasp something visually in my head to understand it. I've been programming for a long time (although I stopped programming for 12 months to being charged with a Computer Hacking offence). I've picked up a lot along the way, but also forgot a lot of concepts I don't use often. When it comes to re-learning the material again I have to start from scratch and face the same learning challenge. I tend to find video training courses are easier for me to grasp. Although a lot of video training courses on programming languages don't go into great detail on certain concepts it makes it easier to understand a concept when I pick up a reference manual or an advanced topic book. One thing I find difficult about learning from articles online, or even from the MSDN reference, is that whilst reading there will be a number of topics in the article that I don't understand so I have to pause reading, go to Google, learn the things I don't understand from the previous article, then go back to the previous article. The thing with this approach though is you tend to get stuck in a sort of dependency hell where you're constantly going to Google and other articles before you start traversing back to the original. I have a nasty problem with pride. It's like everything I learn I want to learn the absolute correct way, and if it's not I don't want to know how to do it at all - again the problem with this is there is no way of knowing if something is correct if you don't know about it in the first place, but it limits me to only learning from credible sources. I fear from learning something the wrong way, or learning bad practices that will get me mocked as a developer. Again this is all part of my Aspergers and not something I can completely control. I will often start a project, and if there is something I need to learn, I will pause it to learn how to do something and come back. The problem is I rarely come back to the project because I'm stuck in a cycle of learning. Again, it's the whole pride thing again. I hate copy and pasting other programmers work without fully understanding what it is their work is doing and if there is any way I can improve on it. Having never worked in the commercial industry I have no idea how it works in the work place. I'm not confident to work in a commercial environment, even though numerous friends who are programmers have said I'm more than capable. Infact, these friends consult me constantly for help on programming topics - yet they work in the commercial environment. But again, I feel this is the Aspergers. I'm trying to find a method of working and learning that boosts productivity and get out of this cycle. Any advice from other programmers or programmers with Aspergers?


title: Best design to create dynamic set of questions(controls ) in silverlight web application?

text: I have around 15 templates (this will grow) and each template will have around 10-15 questions. Each question can have answers in different format like text box, list box, dropdown, radio button etc. I need to show one template in a page, at a time based on the input I am getting. What would be the best design approach for this?   1. Put questions data in database and Create dynamic control?    2. Putting in xml and display using xslt?    3. Creating static set of templates? Or any other approach? I don't have too much time to do this. I am going to use Silverlight for this.


title: Gerrit, git and reviewing whole branch

text: I'm now learning Gerrit (which is the first code review tool I use). Gerrit requires a reviewed change to consist of a single commit. My feature branch has about 10 commits. The gerrit-prefered way is to squash those 10 commits into a single one. However this way if the commit will be merged into the target branch, the internal history of that feature branch will be lost. For example, I won't be able to use `git-bisect` to bisect into those commits. Am I right? I am a little bit worried about this state of things. What is the rationale for this choice? Is there any way of doing this in Gerrit without losing history?


title: "Don't cross the streams" Database access objects independant of the Data tier in an N-Tier architecture?

text: I'm adding functionality to our website which performs long-running processes asynchronously using MSMQ. Doing this ansynch, however means we need to notify users when their requests are completed. Using the command pattern, I created an interface* called INotify and composed that into the message class, so the message processing class can simply call GiveNotice() on the message's INotify object. The first implementation, EmailNotify, was more difficult than expected, as I was surprised to discover MailMessage isn't serializable, but got it going. Now I'm working on a new concrete notifier, DBNotify, which will call a SP of some sort and update a status in the main transactional database. I'm tripped up in that I would like to reuse the DAL architecture we've already created, but INotify is a member of the Model project, which is more fundamental than the DAL. Our hierarchy looks like this: Common > Model > DAL > BAL Here's more detail about our particular n-tiers. Bear in mind, I inherited this, so I'm not looking for suggestions on improving it; I'm wondering how to "break the rules." Common is responsible for all "utility" functions which are used many places in the application, things like accessing configuration settings, parsing strings, non-business related functionality. Model is the repository of business objects, what some folks call data transfer objects, collections of getters and setters. I've added some "smarts" at this layer, but only business rules internal to that object, such as "An item's name must begin with an alphanumeric character." DAL is the data access layer, in theory, all that happens here is model objects are moved into and out of the database. BAL is the Business layer; it encapsulates business rules that govern the interaction of objects (i.e. "A form must have at least two items."). So the INotify interface is defined an abstraction to allow the method of notification to vary independently (i.e. email, TXT, twitter, etc) from the message processing. I have created it at the Model tier, which is independent of the DAL tier, so I can't reuse that code without creating a circular dependency. Has anyone else dealt with a business object whose purpose is to interact with a database, and how do you situate that in your N-tier architecture? Before you tell me to use Linq to Sql, great thanks. This is not a technical question (how do I do this), it's a design question (how should I do this). I think the answer is dependency injection, but I haven't ever used it.


title: `trigger_error` vs `throw Exception` in the context of PHP's magic methods

text: I'm having a debate with a colleague over the correct usage (if any) of `trigger_error` in the context of magic methods. Firstly, I think that `trigger_error` should be avoided _except_ for this one case. Say we have a class with one method `foo()`               class A {         public function foo() {             echo 'bar';         }     }      Now say we want to provide the exact same interface but use a magic method to catch all method calls               class B {         public function __call($method, $args) {             switch (strtolower($method)) {             case 'foo':                 echo 'bar';                 break;             }         }     }          $a = new A;     $b = new B;          $a->foo(); //bar     $b->foo(); //bar      Both classes are the same in the way they respond to `foo()` but differ when calling an invalid method.               $a->doesntexist(); //Error     $b->doesntexist(); //Does nothing      My argument is that magic methods should call `trigger_error` when a unknown method is caught               class B {         public function __call($method, $args) {             switch (strtolower($method)) {             case 'foo':                 echo 'bar';                 break;             default:                 $class = get_class($this);                 $trace = debug_backtrace();                 $file = $trace[0]['file'];                 $line = $trace[0]['line'];                 trigger_error("Call to undefined method $class::$method() in $file on line $line", E_USER_ERROR);                 break;             }         }     }      So that both classes behave (almost) identically               $a->badMethod(); //Call to undefined method A::badMethod() in [..] on line 28     $b->badMethod(); //Call to undefined method B::badMethod() in [..] on line 32      My use-case is an ActiveRecord implementation. I use `__call` to catch and handle methods that essentially do the same thing but have modifiers such as `Distinct` or `Ignore`, e.g.               selectDistinct()     selectDistinctColumn($column, ..)     selectAll()     selectOne()     select()      or               insert()     replace()     insertIgnore()     replaceIgnore()      Methods like `where()`, `from()`, `groupBy()`, etc. are hard-coded. My argument is highlighted when you accidentally call `insret()`. If my active record implementation hardcoded all of the methods then it would be an error. As with any good abstraction, the user should be unaware of the implementation details and rely solely on the interface. Why should the implementation that uses magic methods behave any differently? Both should be an error.


title: What reading is recommended for an experienced Java/SQL Developer moving to Apache/Tomcat, Postgres, Hibernate

text: I am taking over management and development of a website developed in Java using Hibernate and Postgres, running on Apache/Tomcat. I am an experienced Java developer, and have experience in Ruby on Rails, and am expert in Oracle SQL. What is the best documentation for me to read?


title: Which job opportunity is better? The one with the most learning potential or the most teaching potential?

text: I have a little over 10 years of development experience. For the past few years, I've mostly been working solo and although it has its perks, I feel as if it has been limiting my growth. I'm looking to pick up a contract job to get back into a team environment. I've been weighing a couple of opportunities: Company A is using the latest tools (.NET 4, VS 2010, ASP.NET MVC). The interview was fairly technical and I definitely felt rusty and didn't do as well as I had hoped. But I did get the offer. Their development process seems pretty mature. I felt like I could be the "dumbest person in the room" at this job, which I think can be a good thing, from a personal growth standpoint. Company B is a step behind platform wise (VS 2008, .NET 3.5, ASP.NET WebForms). After working with MVC, I find WebForms pretty painful. I also got the sense they have a fair amount of technical debt and they're still working on figuring out a good development process. But at least they're aware of it and working on it. That's an area of interest of mine and I feel like I could come in and provide some good ideas both in terms of development process and paying off technical debt. So these seem like two opposing kinds of jobs: one where I have more of an opportunity to learn technically, and one where I might have a better opportunity to come in and effect change. TL;DR - Which job situation is a better: the one with the biggest opportunity to be a student, or the one with the biggest opportunity to be an agent of change?


title: Fields vs method arguments

text: I just started writing some new class and it occurred to me that I was adding a lot of method arguments that are not strictly needed. This is following a habit to avoid having state in classes that is specific to some method call, rather than being general configuration or dependencies of the class. Doing so means that a lot of methods that could have no arguments end up with one, two or three. I'd like to hear your opinions on what you think of this tradeoff, and how you go about deciding which approach to take in what situation? Since code is often easier to understand than English when describing code, I created a little gist that has both variants in it: https://gist.github.com/JeroenDeDauw/6525656


title: Distributing a proprietary application together with GPL software

text: First, there are many similar questions here, but I found none with direct answers about this _specific_ scenario. If you find a duplicate, please be kind enough to link to the specific answer(s). Also, to avoid question being too broad, consider this specific scenario only. Assume a proprietary Linux application, which links against LGPL libraries dynamically, and does not link against GPL libraries. It also executes GPL'ed binaries from the file system through standard OS APIs though, to perform some actions. Then assume a customized mainstream Linux distribution, with visible trademarked logos etc removed. Then assume a self-contained product, which includes hardware, let's say Raspberry Pi with a bootable SD card with this customized OS. Then it has the proprietary software, which ultimately needs to start when device is switched on. **How to distribute this combination legally?** More specifically, I see (at least) three separate concerns:   1. _In general, how to distribute just the customized Linux distro legally?_ For example, are there tools, or which distros have tools, to download all used default source packages, ready to be included with distribution or archived locally? I think it's sometimes thought, that if you distribute a derivative of a mainstream Linux distro, you do not need to distribute the sources of default packages yourself, or that you can just point to the distro source repositories if someone asks for them, but I'm not sure if this is legally valid way to fulfill GPL obligations.   2. _How to legally include the proprietary software with the distribution?_ For example, is it ok to have the proprietary software already installed in the above mentioned SD? Is it ok to have it uninstalled on the same media, and prompt user for it's installation on first boot? Is it required to have it on separate media or downloaded from the net, and installed from that? What are the limits of "certainly OK" and "certainly not OK", is there any consensus or even legal precedent?   3. _Is executing GPL programs with hard-coded command strings from a proprietary application OK?_ My understanding is, this is OK and does not create a derived work. But if this is an issue, would working around it be as simple as creating an LGPL-licensed wrapper? Or just having the commands in modifiable configuration file (a good idea anyway, regardless of licensing issues)? **Note** : Except for the techincal aspects of first bullet point, I understand that answers here are not any kind of legal counsel, but answers need to be backed up somehow, this site is not for opinion polls.


title: How can I get better at knowing how to write code right off the bat?

text: I have been programming for around 3-4 years now (completed my degree in CS where we were taught the Java programming language). Now my question is: > Do many people find it a difficult task knowing how to write code from off > the bat? What I mean by this is that when I approach a task I would normally look on google to find examples of pre-existing programs to which I would cut and pick sections that would be of use to create my programs rather that think totally logically, knowing what I would need to use from the class libraries to puzzle together to reach my end goal. An example of this is say I was creating an animated GUI. Now for that GUI to work I would usually look at other examples on google and piece together code to reach my desired effect, rather than going to the class libraries and working out exactly what I need to piece together a working program as the libraries can be quite daunting in knowing where to start to find sections that would be useful to you! Just wondering if many others do the same as me or if I'm a minority? Don't know if others find it hard but sometimes I just get confused in knowing what classes I would need or what would be the best way to approach a task.


title: How is your working time distributed between coding and thinking?

text: ...in percentage. For example 60/40 or 90/10 or 100/0. My hypothesis is that the bigger the proportion of time you spend thinking the smaller your code can be as a result (and the less time will be needed to write it down). Think more, write less, in other words. Do you think it is true? As a side note, I think in typical software companies thinking is not part of the culture anyway: you are usually supposed to be sitting there at your computer _typing_ something. You will almost definitely be noticed by your managers if you wander about with a blank look thinking over your next steps with your code. Too bad.


title: Starting C++ Programming (for a C# Developer)

text: I am a C# developer. I am planning to learn C++. (I hope I will be able to leverage the OOP concepts from C#] I have a Windows 7 machine.   1. What are the tools that I need to install for C++ development?   2. Are there any good tutorials for switching over to C++ for a C# developer?


title: What is more standard shortening for "regular expression", "regex" or "regexp"?

text: I want to know what is more standard way to call regular expression in short way. I have seen both, regex and regexp. Google says regex has almost 2 as much hits that regexp, but it depends on way I search it (when I add "regular expression" after the word, "regex" becomes "regex regular expression", the result is differend). And on stackexchange there is regex tag. :) What is more proper way to shorten it? Perhaps it does not matter. :) But thanks for answer.


title: When should use of database be preferred over parsing data from a text file?

text: I was making a Python program to measure the growth of codereview.SE. My approach was to get the "Site stats" shown on the front page and store them on my hard drive. I plan to do this once every day. So far I have made enough to get the stats and append them to a text file. The python script can be viewed on github. The format I am using is the following               22-08-2013          questions 9073     answers 15326     answered 88     users 26102     visitors/day 7407          22-08-2013          questions 9073     answers 15326     answered 88     users 26102     visitors/day 7407      I just ran the script twice to get the format I would be using in the file. Initially this seemed good to me because I would be storing it myself and the format would be the same so it would be easily parsed but not I am not sure. It seems that using a database should be a better here because that way retrieving data should be easier. Just a note, I have never used any database and have no knowledge of SQL, MySQL or any other variants of RDBMS. So this brings me to the question. When should a database be preferred for storing the data over storing the data in a text file? Are there some pointers that I can look for when making decisions about whether I need a database or simple text files? PS: If better tags can be added please do so. I had some doubts about the tags which could be added.


title: Folder conventions

text: is there a resource out there that defines programming folder conventions (e.g. the 'bin' folder)? I googled some but didn't find much. Any relevant discussion on the matter is encouraged. **EDIT** : I'm primarily interested in source management, and I'm not really targeting a specific language (I was hoping to get a reference that might include what's common in the big languages [C, C++, C#, PHP, Python, etc]). Sorry, I didn't mean it to be overly vague.


title: PBI vs User Story

text: Recently an item has been added to the Product Backlog by product owner which says "When I go to login page from x page, I see an error. I want that error to be removed". It seems to me that this is not a use case, and shouldn't be a PBI (Product Backlog Item). However, when I discussed it, scrum master told me that user stories are not PBIs and, a PBI could be a bug report, a task, a user story, anything, and literally any item that should be addressed first. I'm not sure about this. Also I can't find a good definition of PBI on the web. So, my question is, what sort of **things** can get into the Product Backlog as items? Does a product backlog item maps to a user story? Are they the same?


title: Is there a convention for using Yeoman in the context of a larger web framework?

text: I've found Yeoman extremely useful for building out prototypes locally and static mockups that I can share with a team. However, it seems that including its directory structure within the context of a larger framework like Rails, Django, .NET, would be extremely cumbersome, and I can't picture what an efficient workflow might be. There are scads of tutorials that refer to Yeoman and Front End Workflow, but for collaborative development within a larger team environment there seems to be little out there. How to use Yeoman within a framework that is not simply providing a restful API? How to integrate Yeoman within the workflow of a team?


title: How to handle duplicate files in version control

text: Right now we use a very simple system for version control. We have a completely flat master repository for our entire product line. As our codebase grows and we add more products to the product line, this starts to present some serious drawbacks. We'd like to move away from this, especially with out flat style, but there are seemingly fundamental problems with a structured approach, so it'd be great to hear how other people handle these common situations:   1. Your code's got multiple duplicate files in separate locations. For example, you have:              ./somedir/source1.c     ./someotherdir/source1.c      All instances of `source1.c` are the same, so when someone changes `source1.c`, they shouldn't have to make the same change to every `source1.c` in each location where it exists. This is kind of where a flat system works, but it means the structure (with `somedir` and `someotherdir` have to be created to create the correct file structure.   1. On the other hand, if you have something like this:              ./somedir/config     ./someotherdir/config      where both instances of config are unique, and they only share a filename. This presents a big problem in a flat system, you end up doing something like keeping them in a flat repository with unique names, and then renaming them as you recreate the product structure. It seems like this is where a structured file system would make things easier, because there would be no inherent relationship between `./somedir/config` and `./someotherdir/config`. Maybe I am approaching this with a fundamentally bad idea of standard idioms, but as I'm sure you are all aware, when you are making big changes to an existing codebase, it has to be more of a transition than a throw-everything- away-and-start-over situation.


title: Are there advantages to agile practices other than having a working build between sprints?

text: I recently became interested in agile practices in software development and I since have seen a lot of articles point out that these practices allow for reduced overall costs. The logic behind that usually goes like this: if your requirements change, you can reflect this change in the next sprint backlog and this will lead to reduced cost, because designing the new feature and implementing it is close together in terms of time, so the cost goes down, according to the famous rule that the later you need to make a change to your requirements the more expensive it will be to satisfy that requirement. But mid to big software projects are complex. A sudden change to requirements does not mean you won't have to touch other parts of your system in order to satisfy that requirement. In a lot of cases the architecture will need to be modified very significantly, which will also mean you'll need to re-implement all the features that relied on the older architecture. So the whole point of reduced costs kinda goes away here. Of course if a new requirement calls for a new independent part of the system, that's not a problem, the old architecture just grows, it doesn't need to be rethought and reimplemented. And the opposite. If you are using waterfall and you suddenly realize that a new requirement has to be introduced, you can go and change your design. If it requires that existing architecture is altered, you redesign it. If it does not really mess with it but just introduces a new part of the system, then you go and do all the work, no problem here. With that said, it seems to me like the only advantage agile development has is working feature complete builds between sprints, and for a lot of people and prjects this is not critical. In addition, agile seems like it results in bad software architecture overall, because features kinda get slapped one onto another, agile teams only care that a feature works, not how it works. This seems like that when systems grow in complexity with time, agile development practices actually increase chaos in the overall product architecture, thus eventually resulting in higher costs, since it will be increasingly more difficult to introduce changes, whereas waterfall allows you to perfect your architecture before you release anything. Can somebody please point me as to where I'm going wrong here, because obviously a lot of people use agile in production environments, so I must be wrong somewhere.


title: Can GC.SuppressFinalize cause performance problems?

text: I'm being asked to implement IDisposable on objects that are 100% managed resources, which contain no streams, and no large resources. I understand the importance of properly disposing of large resources and unmanaged resources, but what about the other side of the equation? For an object that doesn't benefit from implementing IDisposable (small fully managed objects), what negative impact can GC.SuppressFinalize have, if any?


title: What should I do and know before I start writing C?

text: I have a college degree in CS, and learned about Java and OCaml. Java was used as introduction, and in most CS classed, except the theoretical ones. OCaml was part of learning a different programming paradigm. Since OCaml wasn't a successful entry in my CV, I became a better Java programmer. Also, most jobs are for Java programmers. Later on I learned Golang. I know about the simplifications provided by Java, i.e. garbage collection. I also know, that C doesn't do garbage collection and you have to know when to allocate and when to free the memory. But during my education I was never taught to think in this kind of way. We never learned algorithms in the aspect of memory. I feel like I missed something. So my question is, how deep do I have to dig, before I can start writing passable C code, without having myself put obstacles in my way, because of my own crufty code and deep misunderstanding of relevant issues?


title: How do people manage to write and maintain extremely complex and hard to read code?

text: Reading SQLite source code is IMO mission impossible. Yet it is a usable piece of quite complex software (it's a full-blown embedded database after all) that can be downloaded, compiled and used from others code and it is constantly updated. How do people manage to write and maintain such extremely complex and hard to read code?


title: Handling (many) multiple projects in Git in an enterprise environment

text: One of the advantages of older version control systems such as CVS and SVN in enterprise development is that anyone can connect to source control and see all the projects that the company has. This can make it easier to get a high level view of what kid of development is happening outside your sprint and also keeps everything in one place and easy to find. However, distributed version control systems (Git, specifically) use the repository as their base unit. They work best with one project (or several closely related projects) per repository. This makes repository management more difficult in most enterprise environments where it is not unusual to have more than 25-50 projects to support. As far as I have been able to determine, you have to keep a list somewhere else of all the repos you have. There is software available, like GitHub, that help, but that still is an extra step beyond a single connection string and listing the contents of the repository. What is the best way to deal with the complexity of multiple repositories?


title: When working with contractors/interns how protective should one be of your codebase?

text: We're considering hiring a contractor or intern to work with us on our iOS project but this causes me to get really paranoid that we're giving somebody who doesn't work for the company access to our precious codebase. I mean, I could just give them access to the classes I'd like them to work on but that seems rude and it would make it more difficult for them to develop what we need them to. How paranoid should one be about people running off with their entire application?


title: Testing a file that relies heavily on a static array

text: I have a .c file that basically manages its own little database of people. There are add, remove, edit and search functions among some others. It has a static array which it uses to store the data. What is a good way to go about unit testing a file like this? Because the array is static I can't put in my own data. Should I employ some trickery and make a function that passes out a pointer to the array that gets compiled out in the production version? Should I just test the functions as is and make the data they way I want it via a series of adds and removes? What is the best way to test a file like this?


title: Heavy use of static methods in a Java EE web application?

text: Generally I am asking if this is a norm. The application architecture includes spring and the zk framework. I personally can't help but think this introduces a number of problems. I mean...this is a lot of non synchronized functionality. On top of that we are using an Apache project that, through my browsing of source, appears to use a singleton which has methods that are not thread safe. Changed in a newer version however we are not free to migrate the library at this time. My real question is, is there a justifiable reason for using a large quantity of static methods in a JavaEE application? I was an ASP.NET dev before this and never encountered this. Instincts dictate this is bad architecture but I am unfamiliar with the stack. There are other warning signs such as a lack of conventional generic convention use. Is this the norm? What is foreign in one platform may not be in another.


title: How do I store the OAuth v1 consumer key and secret for an open source desktop Twitter client without revealing it to the user?

text: I want to make a thick-client, desktop, open source twitter client. I happen to be using .NET as my language and Twitterizer as my OAuth/Twitter wrapper, and my app will likely be released as open source. To get an OAuth token, four pieces of information are required:   1. Access Token (twitter user name)   2. Access Secret (twitter password)   3. Consumer Key   4. Consumer Secret The second two pieces of information are not to be shared, like a PGP private key. However, due to the way the OAuth authorization flow is designed, these need to be on the native app. Even if the application was not open source, and the consumer key/secret were encrypted, a reasonably skilled user could gain access to the consumer key/secret pair. So my question is, how do I get around this problem? What is the proper strategy for a desktop Twitter client to protect its consumer key and secret?


title: What is the difference between implementation and realization in Mythical Man Month?

text: In Fred Brook's Mythical Man Month chapter 4 (Aristocracy, Democracy, and System Design) page 49, he says: > As Blaauw points out, the total creative effort involves three distinct > phase: architecture, implementation, and realization ... > > implementer has plenty to do ... designing module boundaries, table > structures, pass or phase breakdowns, algorithms, and all kinds of tools > > ...Meanwhile, on the realization level there is much to be done also. > Programming has a technology, too. If the machine is a new one, much work > must be done on subroutine convention, supervisory techniques, searching and > sorting algorithm. What is the difference between implementation and realization? To me, architecture is related more with specification descriptions or diagrams, whereas implementation is programming/coding which looks like the same as realization.


title: What is "Problem Domain"

text: I have to document my program for a school project and we have section called "problem domain" but I have no idea what to discuss in this section. So the question is: What should be discussed in the problem domain?


title: Mapping variable inputs to user interface elements

text: # Background Developing a system for creating eBooks. The data is highly normalized. The eBook designs are packaged as "themes." The themes can be configured with options for: fonts, colours, some layout, and thematic elements (such as squares for bullet lists could become circles, hexagons, or arrows). The options are tightly coupled to the theme. For example, one theme might only allow the user to select certain fonts, while another theme might not have any thematic elements at all, while all themes might provide options for changing the page number layout, theme colours, and content for the header/footer. Creating eBooks is one example of the general class of problem: > The output depends on configurable options; the available options depend on > a particular constraint set. # Problem When a new constraint set (e.g., eBook theme) is created by a developer, the configurable options must be exposed as user interface elements. The user interface elements could be mapped to the options using a descriptive language, or XML. For example:               <group id=”page-number”>       <group id=”text-attribute”>         <string name=”UserInputPage” default=”# / ##” id=”ui-page” />         <colour name=”UserColourPrimary” id=”ui-colour-primary” />         <select name=”UserSelectBullet” id=”ui-bullet”>           <option id=”ui-select-square” />           <option id=”ui-select-circle” />         </select>         <integer name="UserRandomSeed" min="1" max="50" id="ui-random-seed" />       </group>     </group>     <text language="en">       <!-- Tab name -->       <ref id=”page-number”>Page Number</text>       <!-- Group name -->       <ref id=”text-attribute”>Text Attributes</text>       <!-- Display name for select options -->       <ref id=”ui-select-square”>Square</text>       <ref id=”ui-select-circle”>Circle</text>     </text>      The underlying theme implementation (whether PHP, Lua, Java, or C++) would define, expose, and use variables such as:               UserInputPage     UserColourPrimary     UserSelectBullet      Using XSLT, it would be reasonably trivial to convert the above map into sensible UI widgets that correspond to the variable definition in the XML. By creating this map, developers need not explicitly create the user interface, nor worry about either its layout or menu structure. The developer creates a map of the software's input variables and the computer automatically generates the UI. The UI, in this case will be XHTML, but could be XUL or any other representation. # Comparison Frameworks such as JavaServer Faces are similar to this idea; JSF and ADF issues include:   * Too complex with too much overhead.   * Often tied to specific implementation languages.   * The inputs are bound too closely to their final representation. For example:               <h:inputText value="#{helloBean.name}" ... />     <af:selectOneRadio value="#{bindings.x.hints.inputValue} layout="vertical" />      Vertical "selectOneRadio" radio button inputs should not be hard-coded throughout the application. Instead, the final UI representation should be based on intelligent defaults. In the XML example from the problem section, the UI should generate two radio buttons because there are two inputs in the `select` item. Three items would generate a drop-down. One item would create a checkbox. Similarly, the `<string>` could generate any of:   * input in HTML;   * JTextField in Java; or   * QLineEdit using QT4. Also similarly, the `<integer>` could generate a slider or a simple input field. Validation would be performed elsewhere. With HTML, the final look and feel of the item would be driven by CSS, just as with JSF. # Related Somewhat related:   * Development Approach: User Interface In or Domain Model Out?   * http://en.wikipedia.org/wiki/User_interface_markup_language # Question What approach would you take to create such a variable map definition that can then generate UI code?


title: Configure Jenkins and Tomcat using Puppet on Vagrant

text: I'm playing with setting up my first `Spring` \+ `jenkins` \+ `Tomcat` CI dev environment. For now it's just a test/fun phase, but in the near future I'll be starting new project with my coworkers. That's the reason that I want development environment virtualized and exactly te same on every development machine, as well as on production server. I choosen to use `Vagrant` and to try to write `puppet` scripts that not only install everything, but also configure everything so each of us will have the same `jenkins` plugins, same `jenkins` and `tomcat` login and password, and literally after calling `vagrant up` we are ready to work. What I managed to do so far is installation of stuff needed and port forwarding. My `vagrantfile` looks like this (comments stripped):               VAGRANTFILE_API_VERSION = "2"          Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|            config.vm.box = "precise32"       config.vm.box_url = "http://files.vagrantup.com/precise32.box"            config.vm.network :forwarded_port, guest: 80, host: 8090       config.vm.network :forwarded_port, guest: 8080, host: 8091            config.vm.network :private_network, ip: "192.168.33.10"                   config.vm.provision :puppet do |puppet|           puppet.manifests_path = "puppet/"           puppet.manifest_file  = "default.pp"           puppet.options = ['--verbose']         end          end      And this is my `puppet` file:               Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }          class system-update {       exec { 'apt-get update':         command => 'apt-get update',       }            $sysPackages = [ "build-essential" ]       package { $sysPackages:         ensure => "installed",         require => Exec['apt-get update'],       }     }          class tomcat {       package { "tomcat":         ensure  => present,         require => Class["system-update"],       }            service { "tomcat":         ensure  => "running",         require => Package["tomcat"],       }          }          class jenkins {       package { "jenkins":         ensure => present,         require => Class["system-update"],       }            service { "jenkins":         ensure => "running",         require => Package["jenkins"],       }     }          include system-update     include tomcat     include jenkins      Now, when I hit `vagrant provision` and go to `http://localhost:8091/` I can see `jenkins` running, so above script works good. Next step is configurating `jenkins` and `tomcat` by extending above `puppet` scripts. I'm pretty green when it comes to CI. After wandering around web I've found few tutorials about `jenkins` configuration (here's one of them). I really want to move configuration presented in this tutorial to `puppet` file, so when I spread my `vagrantfile` and `puppet` file between my coworkers, I will be sure that everyone has exactly te same setup. Unfortunately I'm also green about using `puppet`, I don't know how to do this. Any help will be apreciated.


title: Tokenizing Text Held in a Rope Data Structure

text: I am building a text editor which makes use of a Ragel based tokenizer to support syntax highlighting. I am considering the use of a rope data structure to support efficient modifications and undo/redo operations. Is there a standard approach for tokenizing or searching text contained in this type of data structure? Some characters can cause the tokenizer to consume the rest of the stream.


title: Static methods on a web context

text: What happens when concurrent requests call the same static Java method running on an application server? Say that a method run takes 2 seconds and receives 100 calls in a given second. The method itself wouldn't read/write outer state. Could it be better to let it be an instance method instead?


title: Curating projects of deceased friends

text: A very good friend of mine, and an avid programmer, recently passed away. He left nearly 40 projects on BitBucket. Most of them are public, but a few of them are marked as private. I've decided to take on curation duties for the projects rather than leave his work to disappear. If you have been in the same situation, what did you do? Did you open-source everything? Continue development? Delete it all? I'm very interested to hear other people's experiences. There are a few reasons why some of the projects are marked as private (private projects on BitBucket are visible only to invited users and the original creator):   * One of them is an iOS web app that was free in the app store. I've had to remove the app from the store as I'm shutting down his web sites as a favour to his widow. However, I've already made the app public under the GPL v3 (he was a big GPL supporter).   * One of them contains proprietary code. It can't be open-sourced.   * Others are very much work-in-progress. I don't know if he intended to make them into hosted, paid services or if he wanted to give the code away under an open-source licence when they were finished. Here's a list of the private projects:   * Some kind of living cell simulator that uses SBML along with Runge-Kutta and Euler algorithms to do... something. There's a fair amount of code here but I don't know what it does or how far along it is. No docs.   * An accountacy application; it seems to have a solid DB design behind it but there's little code on top of that.   * A website whose purpose is to suggest good restaurants. Built on yii. Seems to have a lot of code but I'd need to set up a WAMP stack to see how far along it is.   * A website intended to host memorials to people who suffered from the same problem he was. Built on Joomla. I'm not sure how much of the code is just Joomla and how much is custom; again, I'd need to get Joomla running to find out. I'd just introduced him to Mercurial and BitBucket. All of the private projects are single commits of codebases he wasn't using version control with/was previously using SVN. I don't have the SVN repositories so I can't see the commit logs.


title: Dealing with inflexible programmers

text: Sometimes programmers who work on a project for long time get inflexible, and it becomes difficult to reason with them. Even if we do manage to convince them, they can be unlikely to implement our suggestions. For instance, I recently joined a project where the build & release process is too complicated and has unnecessary roadblocks. I suggested that we get rid of some of the development overhead (like filling a few spreadsheets) just by integrating defect management and version control tools (both are IBM-Rational tools so integration can be a very easy one-off effort). Also, if we use tools like Maven & Ant (the project involves Java and some COTS products) build & release can be simplified which should reduce manual errors & intervention. I managed to convince others and I'm ready to put in the effort to develop a proof of concept. But the ‘Senior’ developer is not willing, possibly because the current process makes him more valuable. How do we handle this situation without developing friction in the team?


title: Windows Physical Direct Memory Mapping

text: I'm a bit disappointed there is almost no discussion of this no matter where I look so I guess I'll have to ask. I'm writing a cross platform memory bench marking application which requires direct physical address mapping rather than virtual addressing. **EDIT** The solution would look something like the Linux/Unix system calls:               int fd = open("/dev/mem", O_RDONLY);     mmap(NULL, len, PROT_READ, MAP_SHARED, fd, PHYSICAL_ADDRESS_OFFSET);      which will require the kernel to either give you a virtual page mapping to the desired physical address or return that it failed. This does require supervisor privileges but that is ok. I have seen a lot of information about shared memory and memory mapped files but all of these reside on disc and are thus not really useful when I'm trying to make a system dependent read. It is very similar to writing an IO driver although I do no need write permissions to the physical address. This site gives an example of how to do it on a driver level using the Windows Driver Kit: NT Insider: Sharing Memory between drivers and applications This solution would probably require Visual Studio which currently I do not have access to. (I have downloaded the WDK api but it complained about my use of GCC for Windows). I'm traditionally a Linux programmer so I'm hoping there might be something really simple I'm missing. Thanks in advance if you know something I don't!


title: How did Git and Mercurial develop so similarly at the same time?

text: Git and Mercurial follow similar models and have similar terminology. Mercurial's initial release was was only 12 days after Git's. How did these two projects, in initial development at the same time, end up so similar? Does anyone know the history?


title: What algorithms can I use to detect if articles or posts are duplicates?

text: I'm trying to detect if an article or forum post is a duplicate entry within the database. I've given this some thought, coming to the conclusion that someone who duplicate content will do so using one of the three (in descending difficult to detect):   1. simple copy paste the whole text   2. copy and paste parts of text merging it with their own   3. copy an article from an external site and masquerade as their own **Prepping Text For Analysis** Basically any anomalies; the goal is to make the text as "pure" as possible. For more accurate results, the text is "standardized" by:   1. Stripping duplicate white spaces and trimming leading and trailing.   2. Newlines are standardized to \n.    3. HTML tags are removed.    4. Using a RegEx called Daring Fireball URLs are stripped.    5. I use BB code in my application so that goes to.   6. (ä)ccented and foreign (besides Enlgish) are converted to their non foreign form. I store information about each article in (1) statistics table and in (2) keywords table. **(1) Statistics Table** The following statistics are stored about the textual content (much like this post)   1. text length   2. letter count   3. word count   4. sentence count   5. average words per sentence   6. automated readability index   7. gunning fog score _For European languages Coleman-Liau and Automated Readability Index should be used as they do not use syllable counting, so should produce a reasonably accurate score._ **(2) Keywords Table** The keywords are generated by excluding a huge list of stop words (common words), e.g., 'the', 'a', 'of', 'to', etc, etc. **Sample Data**   * text_length, 3963   * letter_count, 3052   * word_count, 684   * sentence_count, 33   * word_per_sentence, 21   * gunning_fog, 11.5   * auto_read_index, 9.9   * keyword 1, killed   * keyword 2, officers   * keyword 3, police It should be noted that once an article gets updated all of the above statistics are regenerated and could be completely different values. How could I use the above information to detect if an article that's being published for the first time, is already existing within the database? * * * I'm aware anything I'll design will not be perfect, the biggest risk being (1) Content that is not a duplicate will be flagged as duplicate (2) The system allows the duplicate content through. So the algorithm should generate a risk assessment number from 0 being no duplicate risk 5 being possible duplicate and 10 being duplicate. Anything above 5 then there's a good possibility that the content is duplicate. In this case the content could be flagged and linked to the article's that are possible duplicates and a human could decide whether to delete or allow. As I said before I'm storing keywords for the whole article, however I wonder if I could do the same on paragraph basis; this would also mean further separating my data in the DB but it would also make it easier for detecting (2) in my initial post. I'm thinking weighted average between the statistics, but in what order and what would be the consequences...


title: What is a "Business Intelligence" Developer?

text: What traits / skills does the Business Intelligence Developer role expect over a traditional Web Developer role?


title: unit/integration testing web service proxy client

text: I'm rewriting a PHP client/proxy library that provides an interface to a SOAP- based .Net webservice, and in the process I want to add some unit and integration tests so future modifications are less risky. The work the library I'm working on performs is to marshall the calls to the web service and do a little reorganizing of the responses to present a slightly more -object-oriented interface to the underlying service. Since this library is little else than a thin layer on top of web service calls, my basic assumption is that I'll really be writing integration tests more than unit tests - for example, I don't see any reason to mock away the web service - the work that's performed by the code I'm working on is very light; it's almost passing the response from the service right back to its consumer. Most of the calls are basic CRUD operations: CreateRole(), CreateUser(), DeleteUser(), FindUser(), &ct. I'll be starting from a known database state - the system I'm using for these tests is isolated for testing purposes, so the results will be more or less predictable. My question is this: is it natural to use web service calls to confirm the results of operations within the tests and to reset the state of the application within the scope of each test? Here's an example: One test might be createUserReturnsValidUserId() and might go like this:               public function createUserReturnsValidUserId() {        // we're assuming a global connection to the service        $newUserId = $client->CreateUser("user1");        assertNotNull($newUserId);        assertNotNull($client->FindUser($newUserId);        $client->deleteUser($newUserId);     }      So I'm creating a user, making sure I get an ID back and that it represents a user in the system, and then cleaning up after myself (so that later tests don't rely on the success or failure of this test w/r/t the number of users in the system, for example). However this still seems pretty fragile - lots of dependencies and opportunities for tests to fail and effect the results of later tests, which I definitely want to avoid. Am I missing some options of ways to decouple these tests from the system under test, or is this really the best I can do? I think this is a fairly general unit/integration testing question, but if it matters I'm using PHPUnit for the testing framework.


title: Usefulness of pre and post order traversal of binary trees

text: This may be very naive, but I was wondering, it the context of binary trees (plain, sorted and balanced), of all the traversal types:   * depth-first pre-order   * depth-first in-order   * depth-first post-order   * breadth-first what's the actual utility of pre and post-order ones? I mean, is there some type and/or configuration of binary tree in which pre and/or post-order traversal would give an(some) advantage(s) over the other two? AFAICS, there are certain types and configuration of binary trees for which in-order and breadth-first might give a certain advantage:   * for a balanced binary tree any depth-first traversal will use less memory storage space as compared to breadth first (eg. for balanced binary tree of 6 or 7 nodes, height is 2 so any depth-first traversal will need to store a max of 2 nodes at any given time, while last level has 3 or 4 nodes so breadth-first traversal would need to store up to 3 or 4 nodes at some point). In this case using in-order traversal uses the least amount of memory and visits the nodes in their natural order.   * for a non-balanced binary tree, if it's close to the worst-case insertion scenario, traversing it breadth-first would use less memory as compared to any of the depth-first traversals. So in this case breadth-first offers an advantage. In-order traversal has the again the advantage of visiting values in their natural order. However I can't think of a situation where pre and post-traversal would give an advantage over the other two.


title: How can I scale spring application context?

text: I'm currently working on a project that need to be dynamically scaled on demand according to the need. My question is about scaling spring context. My web application has a classical multi module core (service, persistence, domain), which use use by client application (webapp, api). The user end application could be clustered, database too, but I don't know how to proceed with spring context. I plan to use Scala Akka actors on Spring service, but I can't be persuaded by the solution of using one context exposed by actors. All of the rest of the application is built with event driven techniques (Spring events are produced by the service layer). I already built different webapps by user profile to scale according to user role (Eg. admin, enduser), however the only thing is Spring Context for business operations. **Question:** Do I have to merge in one single context shared by all client applications or should I initialize one context per application? > ![http://i.stack.imgur.com/0ysv7.png](http://i.stack.imgur.com/0ysv7.png)


Is there a way to play the role of Javascript with any other language like C#? One way came up in my head is, having silverlight installed, using C# instead of Javascript for all the client side scripting (Though C# is not a scripting language). Is it possible? I am not talking about something like GWT(Java) or Script#(C#). Probably the question can be stated as- "With silverlight installed, can I do everything supported by Javascript(like DOM manipulation etc) with C#?" Hope it's clearer.


title: Making Agile and DevOps methodology compatible with PCI requirements

text: Would like to hear from those working in a PCI compliance environment and is practicing agile development and devops methodology, how you maintain compliance with PCI requirements. Specifically, what do you do to address:   * separation of duties between development/test and production   * alignment of continuous integration / deployment and change control   * alignment of agile stories to requirement documentation


title: Does SCRUM usually involve massive overtime?

text: I would like to know if overtime is normal for sprints, or if it is an abuse of this methodology. I feel naive and spineless doing all-nighters if I don't have to. * * * Question is SCRUM specific. Compare methodologies.


title: Who gets source code access with agpl

text: After reading for what seems days on the specifics of GPLv3 and more specifically AGPLv3, I have one question as far as distributing and to whom. If I create a web app (server and client) for a client. They pay me, I hand over all source and ownership. The web app is currently only accessible to employees and contractors; but for the sake of argument, it could be anyone since it's on the internet. The AGPLv3 code is not modified (in this case iText), it is used to generate a PDF from other text (and all this [could] happen in its own project/dll completely separated from the main web app). To who does the source have to be released and shared with:   1. Employees   2. Any user of the app   3. Any non-employee user (clause for distribution within an organization's employees)


title: Model View Presenter for WebForms and Winforms

text: I have a windows forms project using the Model View Presenter pattern and would like to build a web forms project on top of the same MVP structure. The windows forms project knows nothing of ASP.Net Membership and just communicates via the presenter to the model and then to the business layer to do user log on/validation. In the web forms project I have created custom classes that inherit from the built in membership and role provider classes so that I can use the existing user and roles tables as well as integrate nicely with the ASP.Net login controls. However, currently I have had to reference the business layer directly in the custom membership class to validate users, so if I follow MVP, the membership code shouldn't be in the view with references to business layer, it should be in the model but that means referencing System.Web.Security in the model. Is there a way to still use ASP.Net membership with MVP where the webforms and winforms projects are built on top of the same MVP structure? I think I need to pass in the dependency on membership to the presenter and then to the model but can't quite see a clean way to do this. One of the commonly mentioned benefits of MVP is that it allows for building different UI's on top of the same presenters, models, etc. However, in practice, does this really work well or is it best to implement MVP separately per UI technology? Also in relation to the web forms project, is Session state something that the view can just deal with or does this need wrapping with a custom class and interface and passing to the presenter to maintain?


title: Web revisioning and the Accept-Datetime header

text: I have been in the process of generating documentation for an upcoming project. One of the features of the data available in this project is that it will be revisioned (or at least large parts of it will be). I came across the (as I understand it, very new) "Accept-Datetime" HTTP header. This looks absolutely perfect for enabling access to revisioned data using existing interfaces. My question is this. Which return type should be used for:   1. A date received that is before the initial data date (I am thinking maybe 404 with body error description)?   2. Data that is requested using the header, but that does not support versioning (This is the really sticky one for me. Not sure whether an error, or just the normal data is better) Edit: For the moment, in the second case, I am tentatively putting code 406 - Not Accepted


title: Is there a practical guide to Team Software Process?

text: I have seen the Carnegie Mellon site covering TSP. I am looking for a practical, non-academic guide to application of TSP in the real world. I would really like to know, how does TSP reduce bugs?


title: Python 3.1 books still directly applicable to learning Python 2.7?

text: I need to learn Python (v2.7) for my job, and looking for the best intro book for professional programmers. I found (via amazon) that "The Quick Python Book" is the best, but it's for Python 3.1 I know there's a lot of similarities in 2.7 and 3.1, and somewhere read that you can mostly use 3.1 syntax in 2.7 as a good "future practice". Can someone with experience please verify that a book for learning Python3 would still be directly applicable for 2.7? Thank you very much. edit: "The Quick Python Book" is for 3.1


title: How do I prevent useless content load on the page in responsive design?

text: In responsive design, elements are hidden in the page with `@media` queries and `display: none` in CSS. Ok. In my design however browsers that have less than `800px` in width should avoid loading some content at all. When accessed with on a device with more than 800px of screen, the page loads fully. In mobile devices or even on desktop with less than `800px` of width some content is hidden. I want to make the page load faster for low-resolution devices and avoid loading chunks of content that the user will never see. How can I go about this?


title: Bug severity classification issues

text: In a book I have, there is a following classification of defect:   1. Critical : A defect receives a “critical” severity level if one or more critical system functionalities are impaired by a defect with is impaired and there is no workaround.   2. High: A defect receives a “high” severity level if some fundamental system functionalities are impaired but a workaround exists.   3. Medium: A defect receives a “medium” severity level if no critical functionality is impaired and a workaround exists for the defect.   4. Low: A defect receives a “low” severity level if the problem involves a cosmetic feature of the system. To be honest, I do not get it.. For example point 2. What if fundamental but not critical feature is impaired and there is NOT a workaround. The same for point 3: what if no critical functionality is affected but there is no workaround? E.g. optional field in the registration form does not work. No workaround but barely an issue.


title: In good Scrum, when and how should the ideation and story writing occur?

text: It seems like most of the Scrum articles and books that I've read as well as the lectures I've heard have very little to say about the process of creating the items on the backlog. How should a team on a medium or large project (i.e. more than a few team members) go about creating and detailing the items on the backlog? When and how should the ideation and story writing occur? Who should be involved and when? When should team members (other than the product owner, of course) join the process? How much of the team is needed? Note that I am not asking about how a small group would create a user story or other backlog item from ideas. I'm asking more about when and how this ideation and story writing should occur and who should be involved. How can we be more Agile in the ideating/story writing stages?


title: Should website frontends be rendered in client-side or server-side?

text: I've been talking to a colleague that comes from `AS3` to the `HTML` world. He has created a small website (there are apparently no dynamic contents) following the no-flash rules for his very first time, and as if you own a hammer you tend to hammer everything, he decided to create a framework in `JavaScript`, so he could do all "the `AS3` way", with frames, pure X-Y coordinates animation and so, to animate contents. The page would be fully loaded at first, load the vast amount of JS at first and then the page would be accessible without changing url. Views would turn into class methods (where the class represents a section, and the method stands for a view) which append text to a variable that is then added to a div. He defends this situation with many reasons:   * Websites with a high amount of users do this to prevent server overloading.   * It is easier to create new sections.   * It is easier to translate, as translations, texts and almost everything is loaded from a `JSON`.   * You can expand more sections directly from a `JSON` file.   * Everything is smooth, as you don't move to another page. Now, here comes the PHP-loyal developer, which defends the opposite side. I say that views should be split into smaller files but contained in `.php` or even `.html` files, with the business logic in a controller and the view logic in these files. This releases the client, as it can disable `JavaScript` and still show the content without a matter. Of course your server needs to make calculations, but it's just rendering files. Some benefits could be:   * You don't need to load the whole website at first. Webpages loaded faster.   * Your urls are always clean and (in case you did so) friendly, without plugins and working seamlessly in old browsers, as you move to another page.   * Views are kept in independent files, so it is easier to work with them in a team.   * Views are regular `html` files, instead of calls to methods appending strings, so editing them is just natural.   * You still need `PHP` (or any other language) to load data from a server (NodeJS not considered here) And, of course, here comes the real question. **Should views be rendered on client or server side?**


title: Would Task-based programming in C++ require new language standard features?

text: So I saw this video on Youtube with all these C++ masters on GoingNative 2012 : Interactive panel where everybody could ask the questions. This is the video I was talking about: GoingNative 2012 - Day 1 - Interactive Panel: The importance of being native And at time 0:24:00 someone put a very interesting question: > We've been doing concurrent programming for some time using pthreads, using > windows threads, and so on and I'm so happy that C++ and C caught up with > concurrent programming, but it seems to me like it's already behind by five > years or ten years because right now we have we have all these powerful > multicores and the programming of these multicores really should not be > based on threads, it should be task-based [...] and Microsoft has the _PPL > library_ and so on and this is totally not reflecting in the C++ standard. > [...] The only thing I'm afraid of is that the standard could be locked into > threads and sort of make it very difficult to move to _Task-Based > Programming_... Now I'm quite new to these concepts and I'm a little confused. What is actually _Task-Based Programming_. Does this term refer to the same thing that _Lock-Free Programming_ refers to? Are these two equivalent terms or are there any links between them?


title: Do you allow presentation layer code to be deployed outside of a build cycle?

text: For any standard web project you will have server-side code, server pages, JavaScript and CSS files as your code assets. It's pretty standard practice for server-side code to have a test suite and a scripted build process to compile/package/deploy code. Builds usually go on a schedule after code has been tested and validated in a QA environment. On the flip side, we frequently have a requirements to be able to push presentation layer code assets at will. Since they don't require compilation or restarts, this is feasible, but I'm of the mind that this code is coupled to the server-side code and is equally likely to destroy the whole site if it's broken. Therefore, it should never be pushed outside of a build cycle. What is considered best practice for this?


title: Shift bytes through a fixed sized buffer

text: I'm writing an `InputStream` that processes data streams containing "trailer" data. That is, the final **_n_** bytes of the stream is a piece of data that must be handled separately and shouldn't be returned from the `read` methods. My class will maintain an internal byte array of length **_n_** and read data into this byte array from the underlying stream. Data will then be popped from the front of the buffer to service the requests to `read`. Implementing the `int read()` method is simple enough - I just call the same method on the underlying stream, shift my byte array by 1 and pop the result to the caller. Implementing `int read(byte[] b)` is more challenging as it requires shifting a large number of bytes through my **_n_** -byte buffer, whilst popping and collating the bytes from the front. Essentially a "sliding window" buffer, always containing the last **_n_** bytes from my underlying stream. I get the feeling I'm re-inventing the wheel if I start coding this from scratch. Can anyone suggest if I've overlooked an existing `InputStream` or `Queue` implementation that would be well suited to this task?


title: Database in the cloud?

text: Some of my recent clients are asking for remote connections to the office server, for standalone work, etc, in winForm applications. Since the concept of the web is remote connection to a server both of data and resources, it should be possible to place both of this in cloud and have the winForm apps connect to it as if web Apps. As any one tested this, is working like this? Is it fast enough? Is it secure? What is the best cloud host for this type of work ?


title: Deploy multiple emails to email providers, but without showing favouritism

text: We are currently developing a new email deployment system. We have the system currently configured so that it reads a record from the database and loads the email content and deploys it to the target. Now we want to move this over to multiple threads. That is easily done, except we then hit the email providers returning SMTP codes referring to "Too many connections", or "Deferred connection". The solution to this is to have a thread open up a connection to the email provider and deploy _n_ emails and then disconnect. We have currently configured the application so that it will support these _session_ based email deployments. The problem is this, the database table has multiple email addresses in and they aren't grouped by email provider because that will show favouritism. We need to be able to retrieve a set number of, i.e. Hotmail, emails (@hotmail.com, @hotmail.co.uk, @live.co.uk) so that we are reducing the number of connections to Hotmail and reducing the risks of getting the "Too many connections" error. We are at the point now where we have gone round and round in circles trying to get a solution, so I thought I'd throw it out there and see if anyone has any ideas? **EDIT** I would like to stress that this application is not used for spamming purposes.


title: Fast algorithm to search a sorted array of floats to find the pair of floats bracketing an input value

text: I have an array of floats, sorted from smallest to largest, and need to be able to pick out the nearest float greater than or less than a passed input value. This input value is not necessarily present as a value in the array. A naive approach would be to do a simple linear search through the array. That might look like this:               void FindClosestFloatsInArray( float input, std::vector<float> array,                                     float *min_out, float *max_out )     {         assert( input >= array[0] && input < array[ array.size()-1 ] );         for( int i = 1; i < array.size(); i++ )         {             if ( array[i] >= input )             {                 *min = array[i-1];                 *max = array[i];             }         }     }      But obviously as the array gets larger, this will become slower and slower. Does anyone have an idea about an algorithm that would let me find this data more optimally? I've already switched to a binary search, which has improved matters somewhat, but it's still a lot slower than I'd like, and since I'm not actually looking for a specific value that exists in the array, it can never terminate early. More information: The floating point values in the array are not necessarily distributed evenly (that is, the array could consist of the values "1.f, 2.f, 3.f, 4.f, 100.f, 1200.f, 1203.f, 1400.f". I'm doing this operation hundreds of thousands of times, but I can do any amount of pre-processing on the array of floats, if it'll improve the lookup time. I absolutely can change to use something other than a vector to store them, if that'll help.


title: What is the standard /an acceptable period of time to support a particular version of software?

text: Let's say version 2.4.x of Software Package X was released 2 years ago, and was sold up until about 6 months ago. Version 3.1.0 is now out (which fixes a lot of the issues with version 2.4.x), and is the only version actively being sold. In this case, the software is being sold to large businesses, not individuals, so the maintenance effort required is substantial. For how long should the company offer support for version 2.4.x?


title: How do web services for downloading ringtones work?

text: I am interested in knowing more about how ringtone download web services work. I do web programming in java, and i would like to make a web service that sends a ringtone to the user's mobile phone when the user sends a SMS to the web service.   * Is anyone familiar with this type of service?   * What do I need to be able to create somethig like that?   * Does the ringtone system get some profit for each SMS, or does the mobile company get all of it?


title: how could I draw the database transaction commit in the sequence diagram

text: How could I draw the database transaction commit in the sequence diagram? EDIT I want to draw in sequence diagram database transaction commit/rollback process


title: Is this a Proper "Rule" for Identifying the "Big O" Notation of an Algorithm?

text: I've been learning more about Big O Notation and how to calculate it based on how an algorithm is written. I came across an interesting set of "rules" for calculating an algorithms Big O notation and I wanted to see if I'm on the right track or way off. **Big O Notation: N**               function(n) {         For(var a = 0; i <= n; i++) { // It's N because it's just a single loop             // Do stuff         }     }      **Big O Notation: N 2**               function(n, b) {         For(var a = 0; a <= n; a++) {             For(var c = 0; i <= b; c++) { // It's N squared because it's two nested loops                 // Do stuff             }         }     }      **Big O Notation: 2N**               function(n, b) {         For(var a = 0; a <= n; a++) {             // Do stuff         }         For(var c = 0; i <= b; c++) { // It's 2N the loops are outside each other             // Do stuff         }     }      **Big O Notation: NLogN**               function(n) {         n.sort(); // The NLogN comes from the sort?         For(var a = 0; i <= n; i++) {             // Do stuff         }     }      Are my examples and the subsequent notation correct? Are there additional notations I should be aware of?


title: In plain English, what is a database index most similar to?

text: **Setup** : Suppose you are teaching an introduction to Databases class, the students are CS students that have a working knowledge of tree structures, how they can speed up searches, and have probably implemented a few in their lifetime. **Question** : How would you describe the way in which a database uses indexes to search a table for a set of keys? What structure is a database index most similar to? **Bonus** : How does someone write a SQL query where clause to take advantage of the searching capability of the index they design on a given table? Answers should correspond to all database products as a whole. I'm looking for general tips which allow faster searching on all databases. Plain english descriptions please, no code, Big O searching descriptions are fine. This question might be too specific for this site, I considered asking on StackExchange but since I'm requesting a plain english description of a broad concept I thought this site would be Ok.


title: Meaning of MIT license

text: Suppose I use this ActionScript API released under MIT license to build a software: http://www.cove.org/ape/docs/api/   1. Can I sell that software?   2. Do I need to give the source code of my software away?   3. Is anyone receiving my software permitted to resell the software?


title: How to implement a system to hide spammed user content?

text: I'm trying to work out a good way to manage user submitted content and create a method to determine when spammed/poor content should be hidden. The aim is to create comment voting similar to youtubes like/dislike comments and if the score is good show the comments, else they're invisible. **Approach** My thought is to do this via the comments table, you could create two columns rateup, ratedown. For each rate up the comment gets the **rateup** colulmn is: rateup = rateup + 1 and the same for the **ratedown** : ratedown = ratedown + 1 You can then put a clause into your database query to only select comments if their rateup ratio is greater than a number, say -5:               WHERE (rateup - ratedown) > -5       but this would need a lot of voting up and down if a user posts many spam/poor content posts over the site? To strengthen the approach I could add the same two columns to the user table and also update these columns to give an overlal view of the users combined content submission ratio and then display comments like so:               WHERE ((comment.rateup - comment.ratedown) > -5 ) && ((user.rateup - user.ratedown) > -5 )      **Issues** Would this approach lead to instances of users creating mass fake accounts to attack good users and ruin their reputation? What other methods could I use to help hide spammed/poor content without moderators having to manually go through everything?


title: Does Play framework approach necessarily leads to confuse code?

text: I want to start a new project and I just got in touch with Play framework, which has a ruby on rails -like approach that called my attention. I am not an expert in rails, but I think that the productivity in these frameworks are great. I am following some tutorials but I am wondering: Play has the Model, which is used to populate the view but also used to perform DB queries. I find that somehow awkward, but I noticed that rails has also this approach. I wonder if using the same entity for Object Relational Mapping and to populate the view may cause troubles. Here are two examples I can think of:   * My view required only a name and an ID from a big entity   * My view requires information from different entities For the second case, I suppose I would be able to perform a join. But it seems somehow messy: joins in my controller, DB entities being used to populate views. Am I missing some point here? I mean, these frameworks seem great and I would like to use them, but I wonder if my code wound turn into an spaghetti code. It seems to be on the opposing direction of I learned about separation of concerns. If I separate in a layered architecture (models for my views, business logic layer and DAO layers), should I keep using play or should I move to something different like Spring?


title: How can Scrum be adapted to an academic environment?

text: I'm currently working with a professor at my university to develop new curricula for the Software Engineering and Capstone Design courses offered in my college. Up until recently, both courses used the waterfall model exclusively, and thus students spent a majority of their time writing lengthy reports. After much pressure from me, my professor decided to include Scrum in the Software Engineering curriculum this past semester. The first half of the semester was still waterfall-esque, with students writing 40-page design reports and software specification documents. Mid- semester, all teams were required to release a demo of their applications. At that point, the course switched to Scrum, with two 3-week sprints. We are now trying to figure out how to eliminate waterfall altogether and create an exclusively Scrum-based curriculum. Unfortunately, we encountered a few incompatibilities between Scrum and the students:   * Daily Scrum meetings are nearly impossible for students. Even during the class itself, it's inconvenient for students to hold Scrum meetings since the professor is usually lecturing.   * Estimating points/hours is difficult, since students are inexperienced and therefore can't accurately predict how long something will take.   * Scrum works best with full-time, co-located developers, but students are neither. At most, students dedicate 15-20 hours per week to the course, and organizing work meetings can be extremely difficult. Teams can have up to 10 students (and there's always one or two slackers).   * Professors crave documentation! I haven't heard of any Scrum reports—just the backlogs and burndown charts (which I'm not sure will be enough to appease the academics).   * Students often assume that agile means "Jump right in and start coding without looking back". This leads to some of the most horrible code imaginable. Therefore, I'm looking for a way to enforce proper design without requiring a 50-page document or a pile of UML diagrams. Given these problems, how do you think my professor and I can adapt Scrum to function in an academic environment (and should we even bother with Scrum in the first place)? Also, is there still value to teaching the waterfall model? Thanks in advance for any feedback!


title: Difference between "String" and "string" in .NET and which is best?

text: In the .NET Framework, at least in the C# language, we have two "versions" of the string type:   * "string"   * "String" It appears that they are interchangeable, but are they really? If they are not interchangeable, is it generally better to use one instead of the other, and under what circumstances?


title: Is it safe to install TortoiseSVN on production servers?

text: Small shop, few production servers, no test or development servers, all Win2k3 WEB based. There is a need to keep versioned files on one of the machines. Is TortoiseSVN safe against exploitability, does it need extra hardening? Any known license problems, incompatibilities?