Konduit, Inc: The Perfect Database For Business

Choosing an application database can be challenging. If you are trying to reach a decision, but face multiple issues, know that emphasizing on a few elements can help you make a choice. First and foremost, collect your thoughts on what information you would like to track. Next, decide how long you need to keep data accessible before archiving them. Thereafter, be specific about the features, functions, and automation capabilities  you would want your system to have. Additionally, don’t forget that business needs change over time, so set a budget to include periodic modifications.

If unable to make the right decision, get in touch with an expert. To choose the best online database platform, you can rely on Konduit, Inc., the renowned database applications consulting company. Backed by decades of experience of providing structured business applications in an affordable recurring pricing model, Konduit, Inc., is a celebrated company that individuals prefer for the database development of their business. Konduit makes use of Transaction Editor {software as a Service (SaaS), is for small businesses or departments} for database development so as to gain maximum flexibility. Also, Konduit’s database development uses time tested normalization principles that speed-up long range queries.

So, if you want your business database development done right, go nowhere, but Konduit, Inc.

To know more, check the Konduit site

What is a database service?

Database as a Service (DBaaS) may sound like a service business, but really its just another specialized Infrastructure as a Service (IaaS). The CTO Vision article on DBaaS talks about hosting a database management system, but not hosting the application itself. Specifically, the author states “I believe that the reason for the draw is that databases, while critical to your applications, are generally a bit difficult to configure, scale and operate correctly. When presented with the opportunity to off-load that work to a provider, developers jump at the chance.” Developers seek help with databases because they were taught little in school about them and their experience becomes clouded by the many misperceptions about relational databases best described by Fabian Pascal. A good starter article is “Data Management’s Misconceptions” by Joab Jackson.

Software application development that relies on mash-ups of packaged products without professional services to clarify and organize the meaning of data is, well, meaning-LESS. Database management systems and programming languages cannot, by themselves, provide the meaning, integrity, and consistency that gives an application purpose and usefulness. Some ONE needs to be facilitating the discussion of what functionality is needed, and to clarify what the nouns and verbs used to describe a business process mean. Without this clarity of purpose, any software application development will eventually be doomed.

Konduit combines the database development and application development services needed to accelerate the iterative cycle of process adaptation to empower young businesses with a virtual “IT Department”. This service is delivered over an encrypted connection on the Internet, requires little upfront cost, no contracts, and a predictable recurring monthly fee structured to reflect the complexity of the application delivered.  But more importantly, this service includes all the database development, application development, workflow design, hosting services, data stewardship, and end-user support.

The term “database” is filled with mystery, but in reality it’s just about communication and logical organization, something every business needs in order to thrive. The more organized your business information is, the greater chance you will have to settle misunderstandings with your customers and vendors. If you can audit the history of data changes, you can identify and remedy mistakes and know who needs more training.  Database management systems provide a critical part of what is needed.  Information technology is more than hardware and software as commodities. Information technology REQUIRES PEOPLE who understand your business and can rapidly adapt your software-based processes to changes in the marketplace or operations.  To be effective, a freelance web programmer must understand how to use a database management system in the context of a business process automation application.

Free Yourself of Fake News/Gossip

I recently read an article in Bloomberg Businessweek about how Snap Chat employs traditional news media editors to procure public articles in their “Our Stories” feed.  This was refreshing to read as I am careful about the sources I use for my continuing education.  I prefer Businessweek, CBS, and NPR over the popular “social” media that many people use.  It has always been clear to me, and now the world, that Facebook, Reddit, Twitter, Pinterest, and even LinkedIn are simply gossip media with the same audience as the National Enquirer and Star magazines.  Looks like I have some company in this opinion.  Pope Francis’s take.

If you don’t check your sources, you can be easily led to accumulate falsehoods as fact–“alternative facts” as the saying goes.  This idea that spreading falsehoods is bad for society is not new.  It was written thousands of years ago embodied in one of the Ten Commandments.  In case you are unaware, this was written in Hebrew, so having a modern, researched, and precise translation into English is critical so that we don’t lose something in translation.

Translation is in part interpretation, so to provide you with the most accepted semantics of the phrase, I have selected the latest JPS translation according to the traditional Hebrew text in “Tanakh The Holy Scriptures”, which is “…an entirely original translation … into contemporary English.  It is the culmination of three decades of collaboration by academic scholars  and rabbis representing the three largest branches of organized Judaism in the United States.”

Exodus 20:7 and Deuteronomy 5:11 both read “You shall not swear falsely by the name of the Lord your G-d; for the Lord will not clear one who swears falsely by His name.

Of the 611 other rules and laws written in Hebrew, this was one of the Top Ten.  The negative effects of spreading falsehoods is amplified by our modern communications networks.  People die.  Others go to jail.

What does this have to do with Transaction Editor?   One of the built-in features is the audit trail of changes made to data that every Transaction Editor application includes.  I made this a core feature because shipment reconciliation, order processing and billing, and workflow design all demand accountability.   If you cannot correct the cause of mistakes, they will get repeated.   The integrity of the transactional data we use to resolve business transaction differences determines the financial outcome.

This basic tenet, to not swear falsely, also applies to the relationships I maintain with my customers.  If our platform does not fit well with your needs, then I won’t pursue it.  Transaction Editor is a general purpose editing tool with lots of robust features for managing all kinds of information management methods.  Transaction Editor can solve data quality issues and the constant re-engineering of information systems that feed the application development backlog.  But, it cannot do everything.   It cannot deliver a gaming experience.   It cannot analyze terabytes of unstructured data.

Database development is not an exact science.  It requires a free consultation, and possibly a Test Drive, to determine if Transaction Editor is right for your business.  My continuous consulting service IS my solution, so you will never be left to support a complex technology mash-up by yourself. I am dedicated to supporting your business operations, providing superior support in a personal way.  We have no long-term contracts, so we earn your business every month.


Shipment Reconciliation

Big business relies on small business to get the job done.  Critical supply chain processes are often subcontracted to small businesses.  Konduit’s Transaction Editor (TEd) Service has been used by small businesses to keep up with the accounting demands of large clients.  I frequently get the comment that our tracking is better than their large client’s, and this saves them much time and money reconciling differences that would otherwise lead to delayed payments or disagreements.

A farm cooperative has been using Transaction Editor since 2006 to schedule and reconcile deliveries by over sixty member shippers primarily to a single large processing plant. During harvest season from September through July each year, shippers operate 24 hours a day, seven days a week, averaging 1250 shipments per month with a peak of over 5000 shipments in November. The system’s primary Table manages over 185,000 Rows of current and historical data.

The shipment reconciliation process starts with the farm cooperative generating a series of Load Numbers for a particular coop Member.  Included in the generated Load Number is a reference to the Member’s Sales Contract Number plus a sequential Load Sequence Number and a random Load Authorization Code.

When a driver arrives at the Member’s loading scale to receive some bulk material,  Member users log into a Member’s Portal and select from a displayed list of Member-specific Load Numbers.   This link presents them with a form they use to update a shipment record with weight and temperature readings for the Load.  Seal numbers, driver name, and various other information is also updated on the form.   The Portal allows them to print a bar-coded Bill of Lading for the driver to carry.

During transportation, bulk materials can often change in weight, volume, or quantity due to weather conditions, road conditions, vehicle conditions, or driver error.  At the destination, the consignee takes weight and temperature readings with their own systems.  At the end of the day, the consignee produces a file of these shipment measurements from their internal systems for the day’s deliveries and delivers this file to the farm cooperative for shipment reconciliation.

This shipment reconciliation is critical to the accounting that leads to payment or the filing of carrier loss claims.  The cooperative uses Transaction Editor to import the consignee’s daily file and a Report compares it with the Member shipment information.  Several custom Transaction Editor Reports allow the cooperative to  reconcile differences and settle commodity contracts. Their system is quite complex, yet the process of shipment reconciliation is flexible and forgiving.  Every Transaction Editor system includes a data change history audit trail, which forgives human error by allowing the User to investigate and recover from data editing mistakes.

Order Processing and Billing Software

I recently came across a project posted on a freelancer site looking for tight integration with QuickBooks.  The problem is that QuickBooks doesn’t allow data integration, and customization is limited to formatting the appearance of your invoice.  The older products used to use a format called IIF.  When that became public domain, QuickBooks changed their “interchange” format to QIF so that they could legally restrict its proprietary use.  That quickly devolved into another proprietary format, OFX, which was never widely adopted.  More recently, Intuit now allows importing of sales transactions using the Bank Statement interface which accepts any comma-separated (CSV) file with either three (signed) or four columns (debit/credit).   But this is not well known or documented.

QuickBooks provides basic “accounting” functions like receivables reporting, expense journaling, general ledger and financial statements.  But if your business has a complex pricing or estimating methodology, custom order processing and billing software will allow you to automate and codify your business processes without resorting to spreasheets, which is what most people start out doing.  Custom order processing and billing software allows you to print or email invoices, whose revenue transactions can later be imported into QuickBooks for financial accounting.  Importing sales transactions as Bank Statement deposits can be mapped to specific GL account types within QuickBooks by using predefined Descriptions for the transactions.

As your business becomes more successful, you can easily outgrow a  spreadhseet order processing and billing system.   This is where Konduit’s Transaction Editor Service can help you succeed.  For one affordable monthly fee we design the database, develop the application, host, train, and provide continuous support for your staff.  We help you to optimize your workflows that make your business run.

The Transaction Editor Service allows for easy changes and additions to data structures, so you don’t have to know all your needs up front.  We can start with what you do know, and as you work with the system, we refine the structures and workflows as needed.  This removes a tremendous amount of risk associated with communicating requirements of software development projects in advance.

Contact Konduit for a free Solutions Advisor consultation.

Database Design is Easy — Or Is It?

I came across a Request for Proposal from a government entity whose sole purpose was to conform to the data collection standards of a superior government entity.   The following entry is paraphrased for brevity.

Yes/No Questions. The “yes” response and the “no” response each has its own column in the spreadsheet tool.   To specify “yes” for a given question, the user should enter a “Y” in the given question’s “yes” column. To specify “no” for a given question, the user should enter an “N” in the given question’s “no” column.  One question on the form is an exception to this “standard” where there is only one column in the spreadsheet tool for this question, and users should enter a “Y” for “yes”  and “N” for “no”.

This was written by professionals who should be in a different profession.  I struggle with how to explain why this is fubar.  So here goes.

First of all, these are questions on a form, questions, mind you, with only two answers, “yes” or “no”.  Really.  How complicated can that be?  Apparently, very complicated.  If two responses (one in the Yes column and the other in the No column) are allowed, how will one know which response reflects the facts of the respondent?

The noted “exception” should actually be the “standard”.   Most likely, this was written by lawyers to allow for flexible interpretation of the law when enforcing compliance so as not to hurt anyone’s feelings.  Or maybe it exists to confuse the respondent, so they will give up filling out the form entirely to seek out other sources of assistance.  Whatever the reason, the person designing such forms should be in a different line of work because either their brain or their heart are not in it for the right reason.

Why is this important?  Say you wanted to report on the number of respondents who replied to a particular question.  Count the ones who respond “yes”.  Count the ones who respond “no”.  Simple, right?  OK.   Now calculate the percentage of respondents for each.  Well, if some put down “Y” in the Yes column AND “N” in the No column, the sum of the counts of each would be more than the total number of respondents.

Now, image you are a legislative leader and you are reading a compliance report based on these statistics in order to place a vote for a multi-billion dollar extension of the program.  You might see that 65% said “Yes” and 63% said “No” to a particular question on which you would like to base your decision.   Or maybe, the data analyst preparing the “required pie chart” report will recognize the data problem, and fudge the responses so the chart reflects HIS political preferences.   Democracy in action.

I have all the answers, its knowing the right the questions to ask that is the challenge.

For more examples of database development challenges, please visit my favorite website, www.dbdebunk.com.  These design problems are not limited to government entities.  There are plenty of examples in commercial enterprises.

Why People Like Spreadsheets So Much

Business applications in use today are based on a variety of software platforms.  The primary business tool used for maintaining tabular datasets is clearly spreadsheets.  Web-based and thick-client applications offer better data integrity enforcement and change control, but often lack many of the benefits of spreadsheets.  The two primary reasons why users like spreadsheets, and are therefore willing to live without the data integrity enforcement and change control features of software applications, is the ability to sort data on multiple columns and the ability to update and/or add multiple records of data with a single “save” action.  Without these features, the workflow design is less-efficient and wastes end-user’s time

Thick-client application development is accomplished with a variety of grid control components that try to simulate a spreadsheet, but after displaying multiple records, they usually lack an ability to sort on multiple columns without a lot of customization.   Grid controls are usually read-only, require multiple clicks to edit a single record at a time , and hamper the efficiency of process automation.

Web-based applications usually have the same issues as their thick-client counterparts.  The DataGridView is read-only and multiple clicks are required to save changes on multiple records.  Web-based application development also does not typically include any ability to sort on multiple columns.

End-user support for spreadsheet applications is a challenge that can’t be solved by sharing files on a shared cloud space or server hard drive like Microsoft Sharepoint.  Sharing a spreadsheet file does not make it multi-user, and further complicates any attempt to enforce data integrity or manage change control of your tabular data.

Another feature of spreadsheets that drive users to them is the ability to copy multiple rows of data into new rows of data.  No thick-client and only ONE web-based application, Transaction Editor, allows a user to copy existing rows to create new rows of data in a data integrity-controlled environment.

Konduit’s Transaction Editor is unique in that it solves all of these problems with a single platform and support service.  It can replace your spreadsheets with a robust web-based application that offers a workflow design that reduces clicks to complete multi-record updates, allows multi-column sorting, and the ability to copy multiple rows of data to facilitate adding new data records.  Transaction Editor combines the ease-of-use of a spreadsheet with  the robust data integrity and change control management of  a truly multi-user database application.  It even supports multi-table relational data integrity backed by professional database development and design.

Freelance Web Programmer for Small Business

Marketing your services as a freelance web programmer is a tough gig.  But doing so in the United States Software Development Market has become increasingly more challenging.  Optimizing your website for search engines, advertising through Google or Facebook, and sending countless postcards and emails are just no match for the well connected and HR-focused efforts of recruiters.  The old adage “it’s who you know rather than what you know” that drives your success is still true.  But as a provider of application development services, getting you to know me is more than half the battle.

Technology recruiters are extensions of large corporate HR departments.  They work their network of buyers and sellers to earn their healthy markups.  HR Staffers specialize in labor law.  Technology recruiters find candidates with specialized skills.  The problem is, if you are a small business owner without a techno-priest advocating for a specific set of skills, you are left with thousands of technology choices and nowhere to turn for a solution to workflow design and application development.  Oftentimes, a power user or student will have provided a “temporary solution” in Excel or Access that is difficult to support as the business requirements change.

I’ve worked for plenty of recruiters, but recent experiences have demonstrated that disintermediation by web-based freelancer project posting sites are attempting to take market share from sites like Dice.com and Guru.com, who have been around much longer and are more supportive of recruiters.  Some are trying to replace recruiters entirely by claiming to have vetted their applicant’s skill sets.  Unfortunately for them,  recruiting is a local business founded on years of developed relationships.

I don’t know if recruiters ever get calls from small business owners looking for help.  But if they do, I expect that their lower budgets and inability to sign for a minimum engagement of three or more months would prevent a deal.  So the small business owner has to ask a friend, their desktop support provider, their PC/Printer/networking retailer, the local Small Business Association chapter, or local technical college (collectively “small business associates”) for a referral.

What if any of these small business associates were able to refer someone who can help them with a small project and is available within three times zones, can provide continuous support, and costs a mere fraction of the cost of hiring a full or part time application developer?  That is what Konduit does. And we are willing to share with small business associates some of this continuous micro-revenue stream for simple referrals that result in new subscribers.

I have provided on my website TransactionEditor.com a random display of the descriptions of projects we are bidding on.  You can find this at the bottom of the “Portfolio-Your Application” page, under the heading “Posted Projects (RFQ,RFP) Suitable for TEd“. This helps our referrers understand the scope we are focused on.  This should also help small business owners determine whether we might be able to help them.  Along with the description, I rate the quality of the mostly unaltered description as Good, Better, Best:

  • Good = Vague description, but function-related rather than skillset-related.
  • Better = Majority of functionality within TEd capability, but some items require clarification.
  • Best = Well documented functional description of requirements.

With over 20 years of experience, my services also include database development, application development, and workflow design.

Computer Science Degree — Is It Worth It?

Here is a request for a proposal taken from a popular freelance-web-programmer site (spelling errors included):

I am looking for someone who has good experience writing thesis projects. I have a deadline of December 10th to submit my thesis. I am a Computer Science student and almost at the end of my completion of the degree. I have to present a good strong thesis project related to IT which has some programming or implementation involved in it. Below is the brief description which my professor has given me to write my thesis. I am looking for someone who should be good at writing strong tehnical thesis. Money woudl not be an issue for the right person.

I come across these occasionally and wonder what will this person do when they enter the “real world”.  This one is particularly telling with its directness.  Will an employer be able to detect their lack of ability?  Will they be fast-tracked to management because of their pedigree?  Will their spell-checker prevent future exposure to their incompetence?  Or will they eventually resign in shame like the CIO of Equifax?

I believe, for the most part,  there are good schools producing good students.  But seeing things like this worry me.  The general lack of enthusiasm of Millennials and the misplaced emphasis on job placement among colleges (and universities) has degraded our educational system.  And, assuming this is a foreign college, I have much doubt about our ability to fulfill the vacant computer programming positions by increasing the H1B visa quotas.


Why Your Data Loads Take Too Long

During twenty years of doing application development for clients, I have optimized process automation data loads primarily by changing INSERT statements into a BULK LOAD process.  In SQL Server, an external command line utility (bcp.exe) allows bidirectional data streams to flow.  In MySQL, the inward flow is managed by the LOAD DATA INFILE command.

The key to using these BULK LOAD processes is formatting the data properly in files before loading the data.  The best language for parsing large streams of data is Perl.  If you convert your INSERT  statement-generating PHP, Java, C#, VB, etc. to a Perl scrubbing and formatting operation and use the database’s BULK LOAD utility, I am sure you will see a significant increase in the performance of your data loads.  I have used operating system shell scripting languages to tie it all together for automation.

Another factor is the configuration settings of the relational database logging operation.  Most simplified databases do not log transactions because they do not enforce ACID principals.  In fact, most databases, relational or not, are based on straight-forward ISAM principals.  Transactional databases, those that require recovery of transactional (e.g. INSERT) failures, perform extra operations to allow for that recovery.   But what most freelance web programmers don’t know is that this extra activity at every INSERT of a row slows the load process down significantly.  In a data warehouse, this transactional logging can be turned off.  Data recovery is controlled by managing the files used during the bulk load operation.

A BULK LOAD operation in transactional databases will treat the entire load as a single transaction, so breaking up a large set of rows into separate files may make the end-user support more manageable.  The cause of failed file loads can be investigated and dealt with on a more granular level if the files are split into manageable pieces.  What defines “manageable” is determined on a case by case basis by the latency and use requirements of the data.