Tag Archives: TypeScript

Software Development

How to increase heap size for node

Leave a comment

Experienced getting fatal build errors on our build server during ESLint linting of our React project:

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed – JavaScript heap out of memory

<--- Last few GCs --->

[10012:0000000001DA2080] 184320 ms: Mark-sweep 2035.1 (2051.8) -> 2034.3 (2051.8) MB, 2297.9 / 0.0 ms (average mu = 0.114, current mu = 0.001) allocation failure scavenge might not succeed
[10012:0000000001DA2080] 185393 ms: Mark-sweep 2035.3 (2051.8) -> 2033.6 (2051.8) MB, 1068.8 / 0.0 ms (average mu = 0.077, current mu = 0.003) allocation failure scavenge might not succeed

The MB values looked very suspicious in that they were on either side of 2048 MB (2 GB), so suspected our project had run into some sort of limit. And seems there is indeed a max limit of ~2 GB by default on the heap size.

To increase the node heap size limit, set the --max-old-space-size to something more than 2048, e.g. like this on Windows:

setx NODE_OPTIONS "--max-old-space-size=4096" /m

To verify that it actually works (or check what it is by default), this command seems to do it:

node -e "console.log(require('v8').getHeapStatistics().total_available_size / 1024 / 1024)"

Note that the output for some reason always seem to be a bit more than what’s set as the limit. Not sure what that’s about, but anyways… ???????

Software Development

JavaScript: In-browser export to CSV

Leave a comment

This function will turn an array of data into a CSV file and get the browser to “download” it. It should also be wrapping and escaping values properly and, because of the BOM, be read correctly when opened in e.g. Excel.

It’s written in TypeScript ("lib": ["ES2017", "DOM"]), but should be easy to “downgrade” to regular JS if needed…

type GetValue <T> = <I extends T>(item: I) => any;
type FieldName <T> = keyof T;

export interface Columns<T> {
  [header: string]: FieldName<T> | GetValue<T>;
}

const COLUMN_SEPARATOR = ';';
const ROW_SEPARATOR = '\r\n';
const UNICODE_BOM = '\uFEFF';

const wrapValue = (value: string) => `"${value}"`;
const escapeValue = (value: string) => (value || '').replace(/"/, '""');

const makeHeaderLine = <T> (columns: Columns<T>) => Object
  .keys(columns)
  .map(escapeValue)
  .map(wrapValue)
  .join(COLUMN_SEPARATOR);

const makeItemLine = <T> (columns: Columns<T>, item: T) => Object
  .values(columns)
  .map(field => typeof field === 'function' ? field(item) : item[field])
  .map(String)
  .map(escapeValue)
  .map(wrapValue)
  .join(COLUMN_SEPARATOR);

export const exportToCsv = function <T> (data: T[], columns: Columns<T>, filename: string): void {
  const rows = [];

  rows.push(makeHeaderLine(columns));

  for (const item of data) {
    rows.push(makeItemLine(columns, item));
  }

  const csv = UNICODE_BOM + rows.join(ROW_SEPARATOR);
  const uri = `data:text/csv;charset=utf-8;header=present,${encodeURIComponent(csv)}`;

  const link = document.createElement('a');
  link.setAttribute('href', uri);
  link.setAttribute('download', filename);
  link.addEventListener('click', () => link.parentNode.removeChild(link));
  document.body.appendChild(link);

  link.click();
};

Usage

const users: User[] = [{id: 1, name: 'Alice', isCool: true}, ...];
const columns: Columns<User> = {
  'Id': 'id',
  'Name': 'name',
  'Is cool': user => user.isCool ? 'Yes' : 'No',
};
exportToCsv(users, columns, 'users.csv');