Altium Unified Components - Catch 'Em All!

Until recently, the Altium Designer "official" component libraries (known as "Unified Components") were only available to Altium subscribers - they are now however available to anyone by filling out a short survey (no registration or subscription required). After doing this, you receive an email with a download link. This process gets old and tedious very quickly every time you need a new component library... Yes, that's right: Altium does not offer a single download for all the libraries! So I decided to take matters into my own hands and wrote a Python script that uses some XPath and Selenium magic to automatically download all 578 libraries in less than 5 minutes :) This script actually relies on the fact that the library download links are contained in the page source! Thanks Altium!

Requirements:

• Python 3.4.4
• PhantomJS 2.1.1
• Selenium 3.0.2

Usage (Windows):
1. Download and install Python.
2. Install Selenium:

pip install selenium

3. Download the PhantomJS pre-built executable and place in the same directory as script.
4. Run Python script:

python download_all_altium_unified_components.py

I am of course providing the script source for educational purposes only... :)
Any suggestions or feedback on this script are welcome too.

download_all_altium_unified_components.py

from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import re
import concurrent.futures
import urllib.request
import urllib.parse
import posixpath
import os

BASE_URL                = "https://designcontent.live.altium.com/"
START_URL               = BASE_URL + "UnifiedComponents/"
PAGE_LOAD_WAIT_S        = 20
DOWNLOAD_TIMEOUT_S      = 60
MAX_DOWNLOAD_WORKERS    = 8

def wait_for_page_to_load(driver, timeout_s):
    WebDriverWait(driver, timeout_s).until(EC.presence_of_element_located((By.ID, "xUCS_frmItemsView")))

def get_next_page_url(driver):
    try:
        link = driver.find_element_by_xpath("//div[@id='xUCS_frmItemsView']//div[@id='UCS_frmItemsView_TextLabel2']/div/div[1]/following-sibling::a[1][@class='HijaxLink']")
        return BASE_URL + link.get_attribute("href").replace(START_URL, "")
    except:
        return ""

def get_content_urls(driver):
    divs = driver.find_elements_by_xpath("//div[contains(@id, 'UCS_frmItemsView_Fields') and @id!='UCS_frmItemsView_FieldsPopup']")
    return [re.search('ZipURL`(.*\.zip)`[\S+\n\r\s]+', div.get_attribute("title")).group(1) for div in divs]

def download(url, timeout):
    with urllib.request.urlopen(url, timeout=timeout) as conn:
        return conn.read()

# Start WebDriver (choose one!)
print("Starting WebDriver...")
#driver = webdriver.Firefox()
driver = webdriver.PhantomJS(service_args=['--load-images=no'])
print("WebDriver started.")

# Open page
driver.get(START_URL)
# Wait for page to load
wait_for_page_to_load(driver, PAGE_LOAD_WAIT_S)
# Get URLs
urls_all = list()
urls = get_content_urls(driver)
urls_all += urls
print("%r: ZIP download links retrieved from page: %d (Total: %d)" % (driver.current_url, len(urls), len(urls_all)))

# Do remaining pages
next_page_url = get_next_page_url(driver)
while next_page_url != "":
    # Open page
    driver.get(next_page_url)
    # Wait for page to load
    wait_for_page_to_load(driver, PAGE_LOAD_WAIT_S)
    # Get URLs
    urls = get_content_urls(driver);
    urls_all += urls
    print("%r: ZIP download links retrieved from page: %d (Total: %d)" % (driver.current_url, len(urls), len(urls_all)))
    # Get next page URL (if any)
    next_page_url = get_next_page_url(driver)

# Quit WebDriver
driver.quit()

# Write URLs to file
urls_file = open('altium_unified_components_urls.txt', 'w')
for url in urls_all:
    urls_file.write("%s\n" % url)
urls_file.close()

# Download all URLs
error_count = 0
print("%d files will be downloaded..." % len(urls_all))
with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_DOWNLOAD_WORKERS) as executor:
    # Start the download operations and mark each future with its URL
    future_to_url = {executor.submit(download, url, DOWNLOAD_TIMEOUT_S): url for url in urls_all}
    for future in concurrent.futures.as_completed(future_to_url):
        url = future_to_url[future]
        try:
            data = future.result()
            print('%r file is %d bytes' % (url, len(data)))
            # Get filename from URL
            path = urllib.parse.urlsplit(url).path
            filename = urllib.parse.unquote(posixpath.basename(path))
            # Save file
            savepath = "download/" + filename
            os.makedirs(os.path.dirname(savepath), exist_ok=True)
            with open(savepath, "wb") as file:
                file.write(data)
        except Exception as e:
            print('%r generated an exception: %s' % (url, e))
            error_count += 1

# Done!
if error_count == 0:
    print("Finished successfully.")
else:
    print("Finished with %d error(s)." % error_count)

Update 1 (June 25 2017):
Download link for Unified Components archive as obtained from the above script. Enjoy!

Update 2 (December 9 2017):
14 new libraries have since been added by Altium bringing the total to 592 (updated library archive download link). New libraries:
TE_Connectivity_ D-Shaped_Connectors.zip
TE_Connectivity_Audio,_Video_and_High_Speed_Serial_Connectors.zip
TE_Connectivity_Automotive_Connectors_(Car,_Truck,_Bus,_and_Off-Road).zip
TE_Connectivity_Card_Edge_Connectors_and_Sockets.zip
TE_Connectivity_Humidity_Sensors.zip
TE_Connectivity_Modular_Jacks_and_Plugs_(RJ45,_RJ11,_RJ25).zip
TE_Connectivity_Passive_Components.zip
TE_Connectivity_PCB_Connectors.zip
TE_Connectivity_PCB_Terminals.zip
TE_Connectivity_Power_Connectors,_Contacts,_and_Terminals.zip
TE_Connectivity_Pressure_Sensors.zip
TE_Connectivity_Relays,_Contactors_and_Switches.zip
TE_Connectivity_RF_and_Coax_Connectors.zip
TE_Connectivity_Terminal_Blocks_and_Strips.zip  

Update 3 (January 13 2018):
I have slightly modified the script to download all available online content: Unified Components, NanoBoard Examples and Reference Designs (Note: Template Designs seem to only be available via Altium Vault).
download_altium_design_content.py
altium_design_content_20180113.zip [888 libraries, 1.34 GB]

Python + xlwt + xlrd: Auto-formatting specific text in an Excel document

A short Python script I wrote for a friend which highlights (in red) all text in an Excel file which matches a certain regular expression. Tested with: Python 2.7.10, xlrd 0.9.4, xlwt 1.0.0

import xlrd
import xlwt
import re

book = xlrd.open_workbook("input.xls")
sh = book.sheet_by_index(0)

wb = xlwt.Workbook()
ws = wb.add_sheet('Sheet1')

red_font = xlwt.easyfont('color_index red')
normal_font = xlwt.easyfont('')
style = xlwt.XFStyle()
style.alignment.wrap = 1

for rx in range(sh.nrows):
 cell = sh.cell(rx, 0) # Text is contained in Column 0 (first column)
 text = cell.value

 strings = list()
 match_count = 0
 pattern = '\\bwenn\\b|\\bWenn\\b'
 indexes = [m.start() for m in re.finditer(pattern, text)]
 matches = [m.group(0) for m in re.finditer(pattern, text)]
 # Word found
 if len(indexes) > 0:
  prevIndex = 0
  for i, c in enumerate(text):
   # Are we at the start of matched text?
   if i in indexes:
    match_length = len(matches[match_count])
    strings.append(((text[prevIndex:i]), normal_font))
    strings.append((text[i:i + match_length], red_font))
    prevIndex = i + match_length
    match_count += 1
  if prevIndex > 0 and prevIndex <= (len(text) - 1):
   strings.append((text[prevIndex:len(text)], normal_font))
 # Word not found in cell
 else:
  strings.append((text, normal_font))

 # Write data
 ws.write_rich_text(rx, 0, strings, style)

first_col = ws.col(0)
first_col.width = 256 * 100 # 100 characters wide 
wb.save('output.xls')

CMake: Always running the post-build step

As the name suggests, the POST_BUILD option in CMake's add_custom_command() runs after building is complete and as a consequence, if the project target does not need to be re-built (as it has no changes), the post-build step will not be run. However, in some projects you may want to always run the post-build step; for example if you need to copy module libraries (i.e. which do not depend on the main project target but have been modified) to the main target's output directory so the application can run. Luckily there is a simple workaround to achieve this:

1. Add an empty header file (.h) to your project with the following name: always_do_post_build.h
2. Ensure that this file is monitored by CMake by including it in add_executable():

add_executable(${CMAKE_PROJECT_NAME}
  ${SOURCE_FILES}
  ${HEADER_FILES}
  always_do_post_build.h
)

3. Also add the following to your CMakeLists.txt file:

add_custom_target(ALWAYS_DO_POST_BUILD
  COMMAND ${CMAKE_COMMAND} -E echo touching always_do_post_build.h
  COMMAND ${CMAKE_COMMAND} -E touch ${CMAKE_CURRENT_SOURCE_DIR}/always_do_post_build.h
)
add_dependencies(${CMAKE_PROJECT_NAME} ALWAYS_DO_POST_BUILD)

How does it work?

An extra dummy target (ALWAYS_DO_POST_BUILD) is created which depends on the main project target (CMAKE_PROJECT_NAME). Due to this dependency, the dummy target will run before the main project target and ensures that the always_do_post_build.h file (which is monitored by the main project target) is always "out of date" via the touch command. This will trigger the main project target to perform a re-build and thus run the post-build step as required. NOTE: A header file (.h) is used so that no code is compiled in the process.

Update (August 16, 2015)

An alternative to the above is to simply add the following to each module library's CMakeLists.txt file:

# Post-Build Step: Create required directory
add_custom_command(TARGET ${LIBRARY_NAME} POST_BUILD
  COMMAND ${CMAKE_COMMAND} -E make_directory
  $<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/path/to/where/module/should/be
  COMMENT "Creating directory: " $<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/path/to/where/module/should/be
)
# Post-Build Step: Copy library DLL to runtime directory
add_custom_command(TARGET ${LIBRARY_NAME} POST_BUILD
  COMMAND ${CMAKE_COMMAND} -E copy_if_different
  $<TARGET_FILE:${LIBRARY_NAME}>
  $<TARGET_FILE_DIR:${CMAKE_PROJECT_NAME}>/path/to/where/module/should/be
  COMMENT "Copying file to Runtime directory: " $<TARGET_FILE:${LIBRARY_NAME}>
)

XMEGA 32E5: One-shot pulse using the XCL module

As the XCL (short for XMEGA Custom Logic) module found in the new XMEGA E series is a completely new peripheral not found in any previous Atmel chips, documentation and example code is fairly scarce. After taking the plunge and attempting to implement a one-shot pulse as described in AT01084 (Section 4.8.4), I found that the latest Atmel Software Framework (v3.22.0) has forgotten this particular XCL timer/counter operating mode! Perhaps it just isn't as popular as the other modes? But it's incredibly useful if you need a simple way to send a pulse to turn on an external device such as a PC (without using software delays or interrupts in your code).

Anyway, to be more specific on the omission: ASF is missing the One-Shot PWM mode enumeration in the xcl_tc_mode_t enum type (bug report).

Here are the modes listed in AT01084 (Section 4.8):

Not a big deal - we just need to manually specify the MODE mask (0x03) when calling the xcl_tc_mode() ASF function. After that, it's smooth sailing!

In the below example, we will output a logic-low pulse with a fixed duration (after a certain optional delay) on Port D Pin 2.

A few notes before we get into it:

• With a 2 MHz peripheral clock:
• 16-bit XCL timer/counter allows a pulse length of up to ~33.5 seconds.
• 8-bit XCL timer/counter allows a pulse length of up to ~131 milliseconds.
• Pulses can be output on either PD2 and/or PD3:
• 16-bit XCL timer/counter: the pulse can be outputted on PD2 only.
• 8-bit XCL timer/counter: in this case, as we have two separate timers, two independent pulses with different timing characteristics can be outputted on PD2 and PD3.

 

Example:

 

1. Initialize the XCL module and configure to output a one-shot pulse:

xcl_enable(XCL_SYNCHRONOUS);
xcl_port(PD);
xcl_tc_type(TC16);
xcl_tc_mode(0x03);  // MODE[1:0] = 2b'11 : One-shot PWM mode
xcl_tc_source_clock(DIV1024); // Starts timer

2. Set OC0 (PD2) as an output pin and also set inverted so we get a low pulse:

PORTD.DIR = 0x04;
PORTD.PIN2CTRL = PORT_INVEN_bm | PORT_OPC_TOTEM_gc | PORT_ISC_BOTHEDGES_gc;

3. Function to generate the one-shot pulse:

void pulse(uint16_t wait_ms, uint16_t pulse_len_ms)
{ 
    uint16_t _wait_ms = wait_ms / ((1024.0 / F_CPU) * 1000);
    uint16_t _pulse_len_ms = pulse_len_ms / ((1024.0 / F_CPU) * 1000);
   
    // Stop any currently in progress pulses
    xcl_disable_oc0();
    // Start at CNT
    xcl_tc16_set_count(_wait_ms + _pulse_len_ms);
    // Pulse begins at CMP and ends at BOT
    xcl_tc16_set_compare(_pulse_len_ms);
    // Start timer
    xcl_enable_oc0();
    xcl_tc_restart(RESTART_TIMER0);
}

4. Now we simply call the above function with our desired parameters:

pulse(500, 1000); // Delay 500 ms then output 1000 ms pulse on OC0

5. Voila! No interrupts or software delays needed! Easy!