DYI setup and debugging guide

About this page

While assembled Open USB FXS dongle boards will come to the user fully tested and QC-approved, this is not the case for a DIY board. There are various things in doing-it-yourself that can go wrong. Happily, most of those things (with the exception of e.g. ruining a chip or the PCB) are fixable. Moreover, there is a firmware upload step which is not needed in a ready board, but is necessary for a DIY board. All these issues are covered in this page.

While primarily addressed at DIY-ers, this page contains very useful information for ready-made board users, too. You might wish to consult it anyway, to learn more about your board, or in case you face a problem with a ready-made board (a QC-passed board may still fail under different conditions).

A word of warning: this is a lengthy and detailed page, and probably contains too much information to digest at-once. Take your time reading it.

And a final note: this page is full of cases where the board is failing in a way or another. All this is not meant to scare you off building your DIY board. It’s just information to help you, should anything go wrong (I myself wish I ‘d have a guide like this in some of the DIY projects I got involved in).

The schematic

Here is the latest schematic (I know it looks undreadable on this page; try clicking on it to read it). It will be of use in case you need to understand how the circuit works (or why it does not) and chase bugs.

Basic USB knowledge and your board

Open USB FXS is — of course — a USB device. As such, it needs to be enumerated and recognized by your computer’s host controller (HC). By “enumerated”, I am referring to the setup and enumeration procedure defined by the USB standard. By “recognized”, I am referring to the HC being able to map the board’s vendor and product id to the appropriate device  driver.

When a new device is plugged into a USB port, the HC “enumerates” it. During this “enumeration” procedure, the HC queries the device for various parameters and the board reports them back. The HC then decides according to the vendor and product id reported by the newly plugged device which driver to associate with it. Thus, the first step in getting your board to work is to make sure it enumerates correctly. I ‘ll get back to that in a while.

An Open USB FXS board works in two modes, depending on the setting of the switch S2 [note that the setting of S2 does only play a role when the board is being plugged; changing S2 while the board is plugged does nothing].

When S2 is on, the board works in “PICDEM-FS bootloader compatibility mode”. In this mode, the board appears to the host as a Microchip PICDEM-FS device in bootloader mode, which can then be controlled and — most importantly — programmed by means of a Windows program by Microchip called “PICDEM FS USB Demo Tool”.

When S2 is off, the board operates as an Open USB FXS device; it presents to the HC the same vendor id as before (Microchip), but a different product id, specific to the Open USB FXS project [Microchip “lends” its USB vendor id to individual projects and attributes them a distinct product id, under the term that no more than 10,000 pieces are sold — a term by which this project definitely abides :-)]. Note that your PIC does not yet have this firmware loaded. It will be flashed via the “PICDEM-FS bootloader” as explained in the next sections.

Before you start

Before you plug anything into your computer’s USB port, make sure there is no short circuit: measure with an Ohm-meter the resistance between the USB +5V supply line and the ground. It should be at least some kilo-Ohms, possibly increasing as you keep measuring (because of the electrolytic capacitors getting slowly charged). If your board shows signs of a short circuit or a very low resistance, do not plug it in. Try to locate and fix the problem first.

Flashing the firmware (proves that USB works)

While there are many things that must work OK in order for your dongle to operate correctly, the absolutely necessary thing is to make sure that the PIC’s USB operation is OK. At the same time, in order for the board to work as an Open USB FXS device, the respective firmware must first be flashed. For both these reasons, the first thing to do is to switch S2 in its “on” position and try to flash the firmware.

In order to do that, I use the  PICDEM FS tool under Windows. This tool is part of a framework called MCHPFSUSB, which you can download from Microchip using this link. Reportedly there is a utility called fsusb that can do the same job under Linux, but I have not verified it myself. If you choose it, you are on your own. Note also that all initial USB debugging information that I am giving below is also Windows-specific.

After having installed the MCHPFSUSB framework (and rebooted your PC as usual), you are ready: turn S2 on, then plug the board into a free USB port. You should see the usual Windows messages about installing a device driver. After that, you should see the board’s LED flashing quickly on and off (about 3-4 times per second). If that doesn’t happen, look here for more instructions.

If your LED flashes, then your are on the right track! Start the PICDEM FS tool (Start->Programs->Microchip->MCHPFSUSB v2.2->PICDEM FS Demo USB tool), make sure you are looking at the “Bootload Mode” tab, and use the drop-down menu to select your board (presumably, the only one).

Download the latest firmware HEX file from here and store it somewhere on your PC. Then return to the PICDEM FS tool and press the “Load HEX file” button. In the dialog, give the path and the name of the file, OPENUSBFXS-FMWR.hex. In the warning that pops up press “OK” (this will cause an error later, but don’t worry about it; see below).

Finally, press “Program Device”. Some messages will appear about the program flash memory being written. An error stating “Failed to program CONFIG data” will appear. You can safely ignore this (it’s related to the actual PICDEM FS board for which Microchip wrote its utility program and the Open USB FXS board using different PIC configuration settings).

Things can go wrong during this step if your board is unstable. This could be, for example, a loose contact on a PIC pin that is forgotten unsoldered. If things are not rosy, try the same debugging steps as you would if USB did not work at all.

After you have flashed the firmware, quit PICDEM FS, then remove your board from the system. Set S2 back to its off position, and re-plug (no Linux is necessary yet). Your computer should report a new USB device. That’s it! The firmware is there, has taken control of the board and has reported correctly to the HC. Good. Time to move on.

Setting up your Linux environment

The Linux device driver for Open USB FXS works under the dahdi platform (dahdi is the Digium-distributed set of device drivers for a large number of devices that interwork with Asterisk; “dahdi” is the successor name to “zaptel”, the original name of the drivers). Thus, to begin with, it’s a good idea to make sure that dahdi compiles and installs OK under your Linux distro.

The basic requirement to compile the Open USB FXS dahdi driver is a relatively recent kernel (>= 2.6.20). This reqirement fulfilled, the specific Linux flavor that you use is of no importance. I have compiled successfully the driver under Debian (lenny) and Ubuntu, and I got reports of successful compilations under gentoo as well. The dahdi versions that I have tested with Open USB FXS are 2.2.x.y and 2.3.x.y (x and y are sub-minor release numbers that are irrelevant).

So, here are the steps:

  1. Download the dahdi source. For the sake of example, I am supposing that the dahdi source is in /usr/src/dahdi-2.2.x.y
  2. Download the Open USB FXS driver from its svn repository. The driver source should be placed into a new directory named “oufxs” under the drivers/dahdi directory of the dahdi distribution, e.g., into  /usr/src/dahdi-2.2.x.y/drivers/dahdi/oufxs/. To do this in one step, you could for example do this:
    # cd /usr/src/dahdi-2.2.x.x/drivers/dahdi
    # svn checkout   https://openusbfxs.googlecode.com/svn/trunk/LinuxDahdiDriver/oufxs
    oufxs
  3. Compile the oufxs module. From here on, the procedure becomes distro-specific. The most distro-independent method is to issue a “make” from the top-level dahdi directory and including “oufxs” in the subdirs, i.e.
    # cd /usr/src/dahdi-2.2.x.y
    # make subdirs+=oufxs
    On Ubuntu (and possibly others) that use the dkms system, one must fiddle with the contents of the dkms.conffile found in /usr/src/dahdi-2.2.x.x and (a) add “subdirs+=oufxs” to the make arguments in that file (in the line MAKE[0]=…) (b) add an entry at the end of the file for installing the driver. (Added Jul 12, 2011) After the addition of the “checkfmwr” firmware-checking utility, a dependency for libusb-dev (the development version of libusb, with headers necessary to compile programs) was added; you need to install libusb-dev before compiling the module (e.g., on a Debian-derived system, “apt-get install libusb-dev”).
  4. Install the driver under /lib/modules/`uname -r` and do a depmod. Depending on your distro, “make subdirs+=oufxs install” from /usr/dahdi-2.2.x.y might work for you. On Ubuntu, you will need dkms to make the installation. If you install the module manually, it’s a good idea to put it wherever the other dahdi modules are (the location is distro-dependent).
  5. Try to insmod or modprobe the oufxs module. It will require dahdi to be insmod’ed first. It’s a good idea to automate the whole procedure (e.g., some distros provide an /etc/init.d/dahdi script that modprobes the right things in, you could edit that to include oufxs). Edit a new  /etc/modprobe.d/oufxs.conf file, adding a single line “options oufxs lowpower=1 debuglevel=4” (without the quotes). Make sure that you remove any initialization command loading dahdi_dummy automatically from the /etc/init.d/ scripts, or funny things may happen (dahdi_dummy ticks Asterisk at a one-millisecond rate, unfortunately not in synch with the Open USB FXS board(s), something that triggers funny subtle bugs).

Rebooting your system is not really required, however it is a good idea to test if all the /etc/init.d/ scripts are executed OK. After a reboot, make sure that an “lsmod” shows dahdi and oufxs; make also sure it does not show dahdi_dummy.

Obtain a serial number

This is very easy. Just plug your board in a USB port. The very, very first time that you plug your board, the driver will query the PIC on your board for its serial number. Because your board does not have a serial, the driver will write a new one in the device’s EEPROM memory. The driver will not take any further action, and a kernel message (that you can see via ‘dmesg’ or ‘tail /var/log/syslog’) will tell you to unplug and re-plug your board. Do unplug it; as for re-plugging it, read the next section.

Dive in deep waters

Unless you are incurably impatient, please skip this section and read the next ones, then come back here. If, however, you cannot wait any longer to see your new board working in practice, then at this point you can plug your board in (no phone set is necessary yet).

After a moment of hesitation, the board should start flashing its led, initially at a frequency of 1.5 Hz and, later on, a little bit slower. Watch the kernel messages (e.g. by “tail -f /var/log/syslog”) as the driver tries to initialize the board. There will be lots of those.

Whatever it is that you are seeing, make sure to unplug your board after 10 seconds or so, and scrutinize the debugging information at ease, with your (still untested) board unplugged. An untested board may just as well cause some of its components to overheat and burn in case something is wrong. Better be safe than sorry.

If, at the end of the log, you see something like “openusbfxs#1: board setup complete”, then you can congratulate yourself: you are holding in your hands a board that is (more or less) working. If not, don’t panic. Chances are that whatever the error is, it is fixable. Read on to understand the basics of the board operation and to discover how to try to debug it.

The initialization procedure in detail

In this section, I am going to cover the board initialization procedure. This section contains important information for understanding the messages that the driver produces and for debugging your board should something go wrong.

The Si3210 and SPI communication with the PIC

At the heart of your board is a powerful (yet cantankerous) chip, the Si3210 (yes, it is the one that already gave you a hard time soldering it). This chip is a special-purpose digital-to-analog telephony interface processor, which talks to the outer world via three interfaces: an SPI interface, a PCM bus and an analog telephony interface. The SPI interface serves to read and write Si3210 control registers (the only way to control the chip is by setting or reading these registers); the PCM bus carries PCM audio; the analog telephony interface carries, well, analog audio signals. As we are verifying that your board works OK, we shall be dealing with each of those interfaces in turn. The SPI interface is the first we shall deal with.

The initial (SPI) sanity check

Note that, so far, the only thing that has proved working is the PIC’s USB function. The next thing that the driver tries to do is to instruct the PIC to talk to the Si3210 over the SPI interface. To that end, the driver asks the PIC to query the Si3210 over SPI for the contents of a specific control register (DR11) which should have a well-known value (51 decimal). If the driver sees that value, it gives the board a pass for this initial “sanity check”, otherwise it fails it. If your board gives you a failure in this step, see here for more information.

Initialization steps #2 and #3

After sanity-checking the SPI bus, what the driver does next (step #2) is to power down the DC-DC converter (see next section for the converter’s function). It is mandatory to have the converter turned off at this stage and, while usually the board is in its after-reset stage, the converter may have been left in its “on” state if the driver was unloaded and then reloaded while the board was left plugged into the USB port.

In step #3, the driver initializes all “indirect” registers of the Si3210. It’s just a time-consuming step, with no crucial operations. No failures should occur in this stage.

If anything goes wrong during steps #2 and #3 above, this probably means that a register which was just written did not verify. This is again a sign of an unstable SPI bus, and it should be debugged with the same method as for a sanity check failure.

Powering up (steps #4, #5 and #6): cross your fingers!

The DC-DC converter is the part of the board that produces the -65V voltage which is necessary to drive the telephone line. While in principle a very simple component, in practice it has proved to be a common source of trouble, probably due to the relatively high voltages involved (hence “fingers crossed”).

Step #4 is just preparatory and should not give you any warnings. In step #5, DR14 which controls the converter, is told to power it up. After that, the output voltage of the converter is sampled in a loop. You may see this in the debug output messages of the driver, where the sampled voltage is reported every ~10 ms for a maximum of 100 ms (ten iterations, after which, if the absolute value of the voltage is still less than 60V, a failure is reported). Step #6 is a safeguard test: the driver turns off the converter for a short while and measures the sensed voltage as it is dropping. If the voltage falls too fast or too low, this test fails, and a warning is issued (note that a failure in step #6 does not stop the initialization process).

The driver does not give up that easily; in case of a failure, it restarts the initialization procedure from step #1 for two more times until it issues a final error messages and quits. If you see a message reporting a converter powerup failure, unplug your board and then look here for information on how to debug the problem.

Calibrations (steps #7, #8 and #9)

After the converter has powered up, the driver performs a set of calibrations. In step #7 an Analog-to-Digital Converter calibration is performed; step #8 calibrates the current of Q5 and Q6 (if you don’t see these two transistors in the schematic, don’t worry: these are contained within the Si3201); finally, step #9 performs a longitudinal mode balance calibration (and if you don’t know what this thing is, don’t worry, neither do I). In step #9 an additional DC-DC converter calibration is performed.

At the end of each calibration step a success or failure is reported. In case of a failure, the driver issues a warning, then restarts the initialization from the beginning for a couple of times, and after that it finally gives up with an error message. If you get an error at these stages, look here for more information.

Finalization

There is nothing that should really fail in the final steps. During finalization, the driver writes various Si3210 registers to initialize the chip’s interrupt handling, to trim some filter values, to set PCM mode (A- or μ-law) etc. At the end of all this, the driver dumps all direct and indirect register values. These register dumps are of great interest if you are debugging an issue. Presumably if you reached this point, you are not interested in debugging anymore; instead, you are most likely looking forward to plugging your analog phone set in and start running some real tests. This is what the next section deals with.

Operating your board

This section contains all the steps that you should take to get your board to work with a phone set, and finally with Asterisk. Don’t worry if this section looks long, you ‘re hardly ever going to need anything more than a dahdi_cfg command after the first time.

The driver module parameters

Your “oufxs” driver supports quite a few parameters which are related with debugging your board. You may set the value of a parameter by hand (e.g., by issuing a “insmod oufxs.ko parameter=value” to insert the module) or by adding the parameter to the options section of the file /etc/modprobe.d/oufxs.conf (“options oufxs parameter=value“). Personally, I prefer the second method, but it really depends on whether you need a parameter set for a short or for a long time. Here are the parameters:

  1. debuglevel: a value from 0 to 5, in increasing verbosity order. 0 disables debugging messages, while 5 is the most verbose.
  2. retoncnvfail, retonadcfail, retonq56fail: these tell the driver whether to retry forever to initialize the board in case of a failure (in the converter powerup step, the ADC calibration step and the Q5/Q6 current calibration step, respectively). If set to 1 (as it is the default with the current #defines), the driver quits after some two tries.
  3. wpacksperurb, wurbsinflight, rpacksperurb, rurbsinflight: these four parameters govern the driver’s behavior with isochronous USB traffic for audio. They tell the driver how many audio samples to include in a single URB (USB “packet”) and how many URBs to pre-queue. The “w” parameters are for the OUT direction (PC to board) and the “r” ones are for the IN direction (board to PC). The defaults should be OK, but the rule here is that if your machine has a slow CPU, you ‘d rather want to pack more samples per URB. You can experiment freely with these.
  4. sofprofile: setting this to 1 disables the normal operation of the driver and starts profiling your board. We ‘ll see this in more detail in the next subsection.
  5. alawoverride: use A-law instead of the default μ-law. Probably useful in Europe, since A-law is a European standard.
  6. reversepolarity: reverse the polarity of tip and ring. Normally you should not need this.
  7. loopcurrent: the current in mA that should pass through the subscriber’s line. The default is 20 mA and I don’t recommend that you fiddle with that.
  8. lowpower: sets the on-hook and the ring peak voltages to lower values (24 v and 50 V, resp.). This saves lots of energy, and keeps your board cooler (=>it will live longer). I recommend that you set this to 1, unless you have one of these old phones with a rotary dialer and a mechanical ringer.
  9. hifreq: tells the driver to use a higher frequency for the DC-DC converter. The correct value depends on the inductance of L1. The default 1 should not be changed unless instructed otherwise.

There are a few parameters that I will be adding. For example, a careful reader will have noticed that a parameter is missing for setting the driver’s behavior in case of longitudinal mode balance calibration failure. And there are some parameters to fiddle with the audio gain that I should add as well.

The recommended set of parameters for testing are “debuglevel=4 lowpower=1”.

Profiling your board

[Note: I ‘ve realized that I ‘ve changed long ago the code to return the value of TMR3 on every frame, so I ‘ve corrected some of the values below.]
What is profiling, and why/how should you do that? While the explanation of profiling is a bit convoluted, the test is really very easy. So, let us start with the test: unplug your board, rmmod the oufxs module, then re-insmod/modprobe it with sofprofile=1. Plug the board in; this time, after the first few initialization messages, the driver will produce a few lines of hex numbers, and stop there. If everything is OK, all numbers should revolve around the hex value 5DC0 2EE0 (that is, sometimes you might see 5DC3 2EE3, some others 5DBD 2EDD, and some others 5DC0 2EE0).

OK, what are these numbers, then? The answer is that they represent the number of PIC instructions counted by a timer (TMR3) between the first and the third second USB Start-Of-Frame (SOF) commands, then the third second and the fifth third, and so on. These should be always 24,000 (or 5DC0) 12,000 (or 2EE0) but the USB and the board have different clocks and the two clocks may drift slightly, hence the values that are slightly different. The 24,000 12,000 PIC instructions at the specific clock rate of the PIC — 86.66 ns per instruction — amount to 2 1 ms, and the HC emits one SOF per millisecond, so the all odd-numbered SOFs (first, third, fifth, etc.) are rightly 2 1 ms apart. BTW, the board’s firmware compensates for these clock drifts as long as they are not too high. If you get unexpected results, see here.

After you are done with profiling, rmmod the oufxs driver and insmod id again without the sofprofile parameter.

Getting dahdi to configure your board

This is not really a test; it’s a necessary step to get your FXS device ready for Asterisk. You are going to need this every time you re-plug your board. It’s very easy: just edit /etc/dahdi/system.conf and add a line at the end reading “fxoks=1” (or “fxols=1”, or any other signaling scheme you prefer).

What does this do? It instructs the dahdi_cfg command (that you are going to run right afterwards) to set up the signaling mode for your device. What is a signaling scheme? It’s a set of rules by which the FXS device will signal various line conditions by means of electrical signals. For example, when the line “hangs up”, your FXS device may either disconnect the line for a while, or reverse its polarity, or do various other things. The most usual schemes in dahdi are “loop start” (fxols), “kewl start” (fxoks), “earth and magneto” (or “ear and mouth” — e&m), etc.

After selecting a signaling scheme, just execute “dahdi_cfg” (note this command is part of the dahdi_tools package, which you must have installed separately in your system). Normally you should not see any messages. If you execute dahdi_cfg right after you plug your board though, you may get a message indicating that the device is not ready yet. In this case do retry a couple of seconds later.

Note that every time that you unplug and replug your board you need to run dahdi_cfg again. Forgetting to do so will cause subsequent opens to the device file (/dev/dahdi/1 on a system with no other dahdi devices) to fail with NXIO (“No such device or address”). If you see this error, don’t panic, it’s just that you have forgotten dahdi_cfg.

Test your board with (your fingertip and) fxstest

For this step, you will need your fingertip (as a temperature sensor) and “fxstest”. This nice little program will help you with the final steps of testing. Your board must be plugged in, configured with dahdi_cfg and ready to be used.

Before starting, please make sure to touch your board with your fingertip to make sure it’s not too hot. The Si3210 may get moderately hot (~50 °C or ~120 °F), but that’s OK. The other parts of the board should not be too hot. Especially try with your finger Q7. If Q7 is too hot already, unplug your board, then see here.

Connect a phone set to the RJ11 outlet. Then type the command “fxstest /dev/dahdi/1 ring” but don’t press the return key yet. Instead, touch Q7, then with your finger still on Q7, press return. The phone should ring. If it does not, see here. If Q7 suddenly starts getting too hot, unplug your board, then see here. With your finger still on Q7, pick up the phone (take it off-hook, that is) — no need yet to listen to anything. Still sensing the temperature on Q7, leave the phone off-hook for a good 10 seconds. If at any time you feel Q7 getting too hot, unplug your board immediately, then see here.

Everything OK so far? Then, yes, you are allowed to take your finger off Q7. Put the phone on your ear. You should not hear any clicks, hisses, or other similar noises (but some background white noise is to be expected). If you do hear noise, see here.

Whether you hear noise or not, run the command “fxstest /dev/dahdi/1 tones”. This plays some tones (dial tone, ring tone, busy tone, etc.) on your phone. You should now be listening to this series of alternating tones, and you should hear the tones loud and clear. If you don’t hear any audio, see here.

Test your board with Asterisk

The final step is to test your board with Asterisk. I am not going to go into any details here, since there are zillions of pages that describe better than I ever could how to set up an Asterisk system. On a stock Asterisk setup, you will need to add a device in /etc/asterisk/users.conf and a dialplan. It’s a good idea to run the local echo test to hear your own voice.

From this point on, you are a captain on your own board, sailing to whichever destination you choose. Have fun with your new gadget!

Debugging and retrying

There is only one rule about debugging, and it goes “don’t panic!”. About 90% of the errors are fixable. Read throughout this section to get you some ideas about the possible errors and the respective cure. Although I cannot guarantee that every possible failure cause is covered, I think that I am providing insight for some pretty damn obscure cases that gave me a hard time to debug. So, get your meter ready and let us start debugging.

The board does not give signs of life with S2 turned on

This is definitely not a good thing. In the best case, it could mean that you have not installed correctly the device drivers or that you have not rebooted thereafter. You can check if Windows sees your device by invoking the “Device Manager” (e.g., from Start->My Computer (right-click)->Manage->Device Manager). If an unknown USB device appears when you plug the board, then you should make sure that you have installed correctly Microchip’s drivers.

If no USB device appears at all in Device Manager and the board’s LED is dead, check your board around the PIC. Make sure that the PIC, the USB plug, the crystal, CO1 and CO2 are soldered correctly. Check CUSB as well. Make sure that you can measure +5V at the PICs Vcc supply line when the device is plugged. Make sure JLVP is short-circuited (if not, your board is rebooting continuously, and will never make it to enumerate). Make sure that neither crystal line is short-ed to the ground. Make sure that the USB TXD and RXD lines (the two center ones) are not short-circuited between them or to the ground, and that they are connected OK with the respective PIC pins.

If all else fails, use your imagination and any tools that you may have handy (e.g., using an oscilloscope will almost certainly reveal a damaged or faulty crystal). You may try replacing the crystal if you suspect that.

If your debugging findings lead to suspecting a PIC damage (e.g., all power supplies are OK, no short circuits anywhere, no signs of life around the crystal, RXD/TXD paths OK, replacing the crystal does not fix it) you may try replacing the PIC with a new one. However, in that case, you will need to program the new chip using in-circuit-serial-programming (ICSP), so you ‘ll need a PIC programmer and you must add the missing pinhead connector KICSP. Using ICSP, flash the bootloader (not the Open USB FXS firmware!) hex image into the new PIC, then re-try.

The board fails the initial sanity check

This means that the PIC, although communicating properly with your PC over USB, cannot talk to the Si3210 over SPI. There are three possible reasons for that: (1) the Si3210 is damaged or there is another issue with it; (2) there is a hardware issue in the SPI bus lines between the PIC and the Si3210; (3) the Si3210 has not initialized properly (and the cause for that is, quite surprisingly, the PCM bus). Before you end up adopting the pessimistic side of reason #1, do some checks for both reasons #1 and #2: check for shorts and unsoldered pins around the 3210; check the chip’s power supply and ground connections; check if the SPI signal lines are OK and the respective pins of the two chips are soldered, then check for short-circuits. Having crossed out all possible generic errors around the chip and on the SPI bus, then focus your debugging efforts on reason #3, the PCM bus.

You are justified to ask, what the heck does the PCM bus have to do with the SPI bus? In principle, nothing at all. However, as I already said, the Si3210 is a “difficult” chip. As such, it requires the PCM bus to operate correctly in order to initialize properly (and if the chip does not initialize properly, its control registers contain junk values, and this is possibly the reason why your board fails the sanity check — this is not just theory, it’s something that has happened to me quite a few times with my own boards).

To understand the issue in more depth, I must explain that the Si3210 uses the PCLK and FSYNC signals internally for various functions. I cannot be sure about this, but the chip probably does not have any other internal or external clocking source, so these two clocking signals from the PCM bus are very important to it. In particular, the PCLK signal probably clocks simple operations, like moving bits into and out of shift registers, while the FSYNC signal is used to refresh several control registers as the chip resamples various analog values and recalculates some of its internal results (e.g., the driving PWM signal’s duty cycle for its DC-DC converter). A noisy or unstable FSYNC may be the source for many unpleasant surprises in your board.

Debugging the PCM bus (beyond the basic electrical conductivity and short-circuit tests, for which an ohm-meter is sufficient) requires an oscilloscope. You must plug your board in (preferrably with the driver module rmmod-ed) and check for the PCLK signal. It should be a clear 256-kHz square pulse train with a duty cycle close to 50%. Then, you should check the FSYNC signal. This should be a single short pulse every 32 PCLK cycles, that is, a 8-kHz pulse train with a duty cycle of ~3%. If you see noise or instability, try to clean again the area around the pins of the 3210 and the PIC from any soldering paste or flux remainder (this has been the source of the problem in many of my own prototypes). While you ‘re at it, do also check the TXD and RXD lines, to make sure they are clean of noise. RXD had better be checked with a small resistor (~470 Ohms) connected between the board’s “tip” and “ring” telephony outputs, but this is not crucial.

Finally, please note that, in some weird cases, if the PCM bus sort of works, but not perfectly; in such cases, you may pass step #1 while getting strange failures afterwards. In all failure cases, a register dump will follow. If you see all registers containing zeros or 0xFF’s, then this is a good indication that you should look into your PCM bus and fix it.

The DC-DC converter fails to power up

Before we start, please read this note: with the current settings, in the case of a powerup failure the driver leaves the converter in its “on” state forever. This might be dangerous if your board has an issue like a quasi-short. Thus, if a powerup failure occurs, and before you let the board burn by getting yourself absorbed in heavy debugging while some component down the corner is getting ready to catch fire, at least the first few times make sure that you unplug the board early enough and touch all components around the converter to check that no part is too hot.

That having been said, there are four categories of errors here. One, the converter does not work at all; two, the converter sort of works, but does not raise its output voltage fast enough to reach -60V in 100ms; three, the converter works, but a short-circuit or other similar issue in some other part of the circuit keeps the voltage low; four, your USB port does not supply the necessary power (voltage, current or both) for the converter to operate correctly. We shall see how to identify and debug each case separately.

If the converter does not work at all, the driver will always report 0V sensed. You must first ensure that the Si3210 senses the converter’s output voltage correctly (measuring the anode of D1 with a Volt-meter should tell you the truth about this; do also check electrically SVBAT’s connection). If the converter is indeed not producing anything, the cause usually is some bad contact or short-circuit around the converter’s circuitry. Check thoroughly your PCB for forgotten pins and use the Ohm-meter to ensure that all paths are OK. Make sure that the converter gets its +5V supply and that F1 does not cut off. If you cannot find anything else, you may try to debug the situation with an oscilloscope (beware of the high voltages though, some scopes cannot handle -65 V input voltages). Several of my blog posts could help you in interpreting what you are seeing on the scope’s screen.

If the converter does not raise its voltage fast enough, you ‘ll see a series of reported voltages (e.g., 1V, 5V, 9V, 12V) in the driver’s debug output, but the series will not reach 60V in time. Do measure the resistance between D1’s anode and the ground (mind the correct polarity here: you should measure with the positive pole at the ground and the negative at D1’s anode!). If there is no short circuit or a very low resistance, the reason that the converter is not quick enough is likely a faulty FSYNC signal. Look here for an explanation and possible cures.

The measurement at D1’s anode discussed above is the best way to exclude a possible short or similar condition.

The final category (the PC’s USB port does not supply enough power) is the trickiest. The converter needs at least 4.2 V in order to work properly. If you measure the voltage at fuse F1 to be around 4.2 V when the converter attempts to powerup, you are probably in this category. Before putting the blame on the PC’s USB circuitry, you could get yourself a USB extension cable (type A male to type A female), and strip (carefully) the protective cover around the four wires somewhere in the middle of the cable. Make sure you identify the +5 V supply line (or the GND, but not both) and cut it off. Cut the line and intercept a milliamp meter. Measure the current that your board consumes when the converter powers up (with no phone connected). If the current is more than 350 – 400 mA , then you should look more carefully at your board for short circuits or other similar issues. You could try removing R21 temporarily (or de-soldering pins 4 and 5 of the 3201); if that fixes it, then the issue is likely at or around the Si3201; if it does not, the issue is likely around the converter circuitry (could be for example a damaged C9).

ADC, Q5/6 or longitudinal mode balance calibration fail

An error in step #7 (ADC calibration) is quite unusual. To tell the truth, I don’t know what would cause such an error. Errors in Q5/Q6 and longitudinal mode balance calibration steps are an indication that some problem exists with the Si3201 (not the 3210). If you see such an error, make sure that the Si3201 is soldered correctly and all paths are OK (continuity and no shorts). If nothing is found and these errors persist, then it is probable that the 3201 has been damaged and needs to be replaced.

An error in the final DC-DC converter calibration results in a cryptic message about dr107’s value, but does not stop the initialization process. This error means that your board is working in a marginal power state, in which the converter cannot be calibrated to work any better than as instructed by the driver-supplied default values. Measuring the USB-supplied voltage and current to cross out the possibility of a weak-power USB port would not hurt.

SOF profiling returns unexpected results

SOF profiling might fail once or twice, or report that the board is not responding. Don’t worry if this happens, just re-try a couple of times and you should get three lines of output with no errors.

If you see values completely different than 5DC02EE0, or if the drifts are high (e.g., you see 5DD02EF0 instead of 5DC02EE0), then there is a serious issue with your board’s clock. First try reflashing the firmware using the board in bootloader mode (S2=on) and PICDEM FS.

If the problem persists, try your board on a different PC. It may be the case that your PC’s USB port is indeed clocking the USB bus too fast or too slow. If you try on another PC and the problem shows up there too, then try replacing the 20 MHz crystal.

Q7 gets too hot

“Q7 gets too hot” means it gets so hot that you instinctively want to pull your finger off (and shake it in the air to cool down). This is an indication that the DC-DC converter is not working properly. The first thing you should do is identify the exact conditions under which this happens. Does it happen when the phone is off-hook or ringing or all the time (after the converter powers up, of course)? Make sure to run your tests in short intervals and unplug the board afterwards, not exceeding 10-15 seconds or so for every test, or else you risk burning Q7.

If Q7 is too hot all the time, even with no phone connected, this is an indication of either a short (measure the circuit around D1 for this) or a problem with the DCFF path. For the latter, make sure that the 3210’s DCFF pin is soldered correctly and the path from that pin through C10 to Q7’s base is also OK.

If Q7 gets hot quickly when the phone gets off-hook or is ringing, this is an indication that the converter is working under marginal power. See my blog post “Power games” for more information on this. In this case, I am sorry, but I cannot be of much help, since the culprit is the PC’s USB power supply. Do repeat the measurements of the “Converter fails” section to verify the cause of the problem.

Even if there is not enough power, not everything is lost: you can still try to power the converter part of your board from an external source. Use a 5V supply (a mains adapter for charging phones is fine, as long as it can deliver ~450 mA of current at 5 V DC). Unsolder F1 so that it connects only to the converter side (C25/C14), and connect the external power supply to the free end of F1 (positive pole) and the board’s ground; then, repeat all the testing from scratch.

The board produces noise

There are various possible causes for this. First try to identify the type of noise. Does it consist of “clicks” or is it a series of clicks? If yes, does this series of clicks get faster or vanish with time? Is it related to the off-hook operation? That is, does it have a pattern that repeats after taking the phone off-hook (or on-hook and back off-hook)?

Noises like clicks that relate somehow to the off-hook operation are indications that the Si3210 does not like the operating conditions of the DC-DC converter and is attempting to restart it. Most of the times, this is also causing Q7 to get hot. Make sure you don’t let it burn.

You may check the voltage sensed by the chip by taking the phone off-hook, then issue a “fxstest /dev/dahdi/1 regdump” command and study the value of direct register 82. Multiply that value by 0.376 and you ‘ll get the voltage produced by the converter. Repeat the measurement several times to spot low values (e.g., 8V or so). Such low values cause the converter to restart. In this case, I suggest you repeat the measurements of the “Converter fails” section.

In contrast to clicks, a “hiss” is an indication of something going wrong at the output stage. Make sure that C5 and C6 are OK. These two caps decouple much of the high-frequency digital noise caused by the swithcing nature of the DC-DC converter.

A beeping sound is an indication of the converter oscillating or restarting itself at a frequency near 1 kHz. Make sure you have not fiddled with the “hifreq” parameter (unless instructed to by me). Recheck the circuitry around the converter and remove any shorts or paste leftovers.

You should differentiate between the noises described above and occasional clicks. The latter (random clicks, every now and then) might be due to lost USB timeslots (that is, 1-ms timeslots during which your PC’s USB HC driver is not fast enough to schedule and transmit a URB). Normally, unless your system is extremely highly loaded or has an old, very slow CPU or is running as a virtual machine, this should not happen. If it does, though, it can be fixed by increasing wpacksperurb, wurbsinflight, or both.

Phone not ringing or no audio at all

If “fxstest yourdevice ring” does not ring the phone there can be two reasons. One, your USB port does not supply enough power, so when the board attempts to pump power to the phone, the board reboots (you can check if this is the case by watching the syslog messages). Two, there is an issue with or after the Si3201. This second case requires a bit more debugging.

Before continuing, it’s a good idea to check your circuit around and after the Si3201 (visually and electrically). Most probably, if there was a problem with the Si3201 itself, the q5/q6 and LMB calibrations would fail. If they did not, this is a good sign, and you should check for a forgotten or mis-soldered L3/L4 or something similar.

If everything looks OK, then you could plug your board with your phone off-hook and try to hear the “click” as the converter is engaged and raises its voltage. If this is heard, then this is again a good sign, otherwise chances are that you have a damaged Si3201.

If nothing suspect was found, try reloading the module without setting lowpower to 1 (this increases the power dissipation of the board and you should probably recheck for the usual supsect components getting too hot). If this doesn’t fix it either, then double-check the ringer muting switch on your phone (you never know :-)).

4 Responses to “DYI setup and debugging guide”

  1. Brecht Van Wanseele Says:

    Hi angelos,

    I’m having porblems with insmod or modprobe of oufxs.
    Everything compiles fine but when I do a “lsmod”, only dahdi is shown, not oufxs.
    In the guide it says that you can automate it by changing the dahdi script, wath needs to be changed exactly ?

    kind regards,
    Brecht Van Wanseele

  2. Angelos Varvitsiotis Says:

    Hi Brecht,

    As you are describing it, it seems that you have succeeded insmod’ing or modprobe’ing the oufxs module manually. If this is accurate, however you don’t see the oufxs module after a reboot, you should edit the dahdi initialization script or a configuration file that tells the script which modules to probe. The script is usually /etc/init.d./dahdi and the configuration file, if it exists in the dahdi version you use, is /etc/dahdi/modules. It suffices that you edit /etc/dahdi/modules and add oufxs in there; if /etc/dahdi/modules does not exist, you should add “modprobe oufxs” by hand somewhere inside /etc/init.d/dahdi.

    Cheers,

    Angelos

  3. jack Says:

    Hello, May I want you explain to me the 3, 4 and 5 step of “Setting up your Linux environment” paragraph more ? I’m new to ubuntu and can’t do this steps completely. I found the dkms.conf file and opened it with terminal but don,t know where I must add subdirs+=oufxs and what I should add to end of file.and etc.
    Thanks,
    Best Regards.

  4. Angelos Varvitsiotis Says:

    Hi Jack,

    I have received questions like this before, so I think the best is to upload a dkms.conf sample on my google project repository or to include some code sample on my blog. Currently I am working on re-testing my linux setup procedure for Ubuntu 12.0.4LTS, and after that a new post should be coming. I am asking for your patience for a couple of days. In the meantime, please find a sample below (it is from Ubuntu Workstation 10.0.4LTS):

    # Items below here should not have to change with each driver version
    PACKAGE_NAME=”dahdi”
    MAKE[0]=”make modules KERNVER=$kernelver MODULES_EXTRA=’opvxa1200 wcopenpci dahdi_echocan_oslec’ SUBDIRS_EXTRA=’../staging/echo zaphfc/ oufxs’ ;make;make firmware-loaders;echo : > drivers/dahdi/vpmadt032_loader/.vpmadt032_x86_32.o.cmd;echo : > drivers/dahdi/vpmadt032_loader/.vpmadt032_x86_64.o.cmd;make”
    #Do not make clean, because Sangoma drivers need Module.symvers
    CLEAN=”/bin/true”
    #CLEAN=”make KERNVER=$kernelver clean”

    …. (and in the end):

    BUILT_MODULE_NAME[35]=”oufxs”
    BUILT_MODULE_LOCATION[35]=”drivers/dahdi/oufxs”
    DEST_MODULE_LOCATION[35]=”/kernel/drivers/telephony/dahdi/oufxs”

    AUTOINSTALL=yes

    Cheers,

    Angelos

    Cheers,

    Angelos

Leave a comment