Skip to main content

Ideas for KiCAD library crowdsourcing

As part of an on-going discussion on the KiCAD devleopers mailing list about library crowdsourcing, I wanted to put together more detail about the issue. I've been using KiCAD for a handful of months and was unaware of some of the organizations, such as CircuitHub and KiCadCloud, that have been working to improve the situation until this discussion.

Background

PCB development flow

A typical circuit board development process in KiCAD looks something like:

Schematic (eeschema)

  1. Design a circuit
  2. Locate components (or create the component symbols)
  3. Add components to the schematic
  4. Connect components on a schematic using the component symbol

Map component to footprint (cvpcb)

  1. Choose the specific footprint for each component in the schematic (or create the necessary footprints)

Layout (pcbnew)

  1. Place components
  2. Route the signals on the board

The problem

During schematic step #2 and the component mapping process, the highlighted steps above, any component that doesn't already exist has to either be created or downloaded. While there are new parts being introduced regularly there are also a lot of people using KiCAD (and other eda tools) that are creating the same part symbols and footprints.

A solution

Make it easier for users to share and use part symbols and footprints so other users could use their part models rather than creating their own.

Current approaches

There are a few different online solutions where you can get or share components.

circuithub.com



CircuitHub uses a novel approach. By combining their site with DropBox, circuithub will add to your DropBox the parts you've selected via the web interface. By using the filesystem and not attempting to interface with the application directly, CircuitHub is able to interface with Altium, KiCAD and EagleCAD.

kicadcloud.com



KiCadCloud.com allows users to download parts libraries as well as upload them if they've created an account on the site.

kicadlib.org



Kicadlib.org is a site where users can have their libraries added as links from a main page. There are quite a few parts available through the libraries. New libraries appear to be submitted via email to the maintainer.

If the part you are looking for isn't available through any of those online services, you'll have to create it yourself. Creating parts can be time consuming and error prone. How can we improve this situation by adding value to the user?

What the user wants to be able to do

  • Find a well created component symbol
  • Find a compatible and correct footprint
  • Contribute:
    • Feedback on symbols or footprints, maybe voting or text comments or bug reports
    • New symbols and footprints
    • Modified symbols or footprints
  • Store used symbols or footprints in local files for version control, to work without a network connection and to mitigate concerns that symbols or footprints might go away.
  • If client/server:
    • Ability to interact with the server without an account, in a read-only (non-contributor) mode. Simplifies the initial adoption process.
    • Web interface
      • Browse and view entries in the library
      • Add comments to parts
      • Search by keywords
    • Receive email notification if someone comments on their part or reports an issue.
    • The ability to run their own instance of the server
      • Companies could use this to share parts among developers or groups
      • Prevents vendor lock-in

Helpful features for server operators

Assuming a client server approach, some features that have been beneficial to server operators are:
  • Ability to have roles such a maintainers with permissions to help curate the library
    • Could use hard defined roles or a reputation system like the one found on StackOverflow
  • Runs on off the shelf modern website software
  • Runs on cloud instances, makes deploying new servers easy
  • Licensing options per-part
  • Ways to flag entries as bad
  • Email notifications if parts have been flagged
  • External user authentication to remove the necessity to manage user accounts on the server

Communication API vs. Server implementation

The communication api and the server would be treated as separate entities. For instance the api may be jointly developed with the community, CircuitHub, and KiCadCloud's participation. Each of these organizations may have their own sever implementation with different features but they would remain inter-operable through the common api.

Implementation notes

  • Client/server approach
  • RESTful API using json
    • Simplifies the server API
    • Possible API including:
      • login / logout
      • part search
      • get part by hash
      • get list of hashes for user account
      • put new part
      • put new part version
      • put part feedback
  • Fully documented API
  • Open source server side code, GitHub or other online hosting for the server side code
  • Possible cooperation with CircuitHub on the API
  • Create an official KiCAD developer blessed master server. Provides a trusted source for users and a pre-installed cloud library to get users started with the approach.
  • Hash based data tracking
    • Based on how git uses hashes
    • Hash provides a unique identifier for each model.
    • Hash generated from all data elements that are critical. The model data plus author may be a good start.
    • Detect identical parts.
    • Version control can use the hash
  • Database tables
    • Data elements for each model
      • Author
      • Creation date
      • License
      • Model content (symbol or footprint)
      • 3D model
      • Comments
      • Datasheet url
      • Votes
      • Server settings
    • User settings settings
      • User roles etc

Thoughts on leveraging existing tools

  • CircuitHub / Dropbox Approach
    • Dropbox is a proprietary tool
    • How is part versioning handled?
    • Concern for vendor lock-in?
    • How is a new part added from the client/tool side? A helpful step-by-step guide on the CircuitHub blog shows server->client data flow.
    • Using the file system is more transparent to applications but seems to make the server side more complex while limiting the client side functionality. Overall the approach is simpler but may be less extendable and more work for the end user.
  • Could the Dropbox approach be abstracted to use git for the data exchange?
    • Leverages an existing well tested tool (git).
    • How to handle permissions on pushes?
    • Would each user have their own branch?
    • How are part changes pushed from client to server?
    • How would file naming be handled? Is the naming arbitrary and could be the name of the hash value? Odd/long filenames would make it difficult to interact with the parts library via the filesystem.
    • Server could interact with each users branch in response to web interaction.
  • Could git be leveraged internally on the server side?
    • Use git as a general purpose data store on the server side.
    • How to attach things like votes, attributes like datasheets etc via git? A database would seem more natural for that.
    • Git could handle versioning but how would we decide where to place a part in the filesystem?
    • Could use git's internal data store directly.
Labels:

Comments

Post a Comment

Popular posts from this blog

Debugging an imprecise bus access fault on a Cortex-M3

This information may apply to other cortex series processors but is written from practical experience with the Cortex-M3. Imprecise bus access faults are ambiguous, as noted by the term "imprecise". Compared to precise bus errors, imprecise errors are much trickier to debug and especially so without a deep understanding of arm processors and assembly language. Imprecise and precise flags are found in the BusFault status register, a byte in the CFSR (Configurable Fault Status Register). BusFault status register bits The definition for imprecise and precise bits is: [2] IMPRECISERR Imprecise data bus error: 0 = no imprecise data bus error 1 = a data bus error has occurred, but the return address in the stack frame is not related to the instruction that caused the error. When the processor sets this bit to 1, it does not write a fault address to the BFAR. This is an asynchronous fault. Therefore, if it is detected when the priority of the current pr
5 comments
Read more

Graco Swing By Me - Battery to AC wall adapter modification

If you have one of these Graco battery powered swings you are probably familiar with the cost of C batteries! The swing takes four of them and they only last a handful of days. I'm not sure if the newer models support being plugged into the wall but ours didn't. If you are a little familiar with electronics and soldering, here is a rough guide on how you can modify yours to plug in! I wasn't sure how exactly to disassemble the swing side where the batteries were. I was able to open up the clamshell a bit but throughout this mod I was unable to determine how to fully separate the pieces. I suspect that there is some kind of a slip plate on the moving arm portion. The two parts of the plastic are assembled and the moving arm portion with the slip plate is slid onto the shaft. Because of the tension in that slip plate it doesn't want to back away, and because of the mechanicals that portion of the assembly doesn't appear accessible in order to free it. I was
Post a Comment
Read more

Memory efficient queuing of variable length elements

In embedded environments memory can be a critical driver of the design of data structures and containers. Computing resources have been expanding steadily each year but there are still a wide range of systems with far less than a megabyte of memory. On systems with tens of kilobytes of memory, structures are often designed to be compact to maximize data density. Rather than splurging on memory aligned elements that would be faster for the processor to access, a developer will typically use types with minimal sizes based on the known range of values that the element is intending to hold. Fixed sized buffers At my day job a fixed size pool of messages was implemented to hold message data. While this achieved one design goal of using statically allocated buffers, avoiding dynamic allocations that might fail at runtime, it isn't efficient if there is a wide range of message sizes. It isn't efficient because each message uses a message buffer. With small message sizes the buff
2 comments
Read more