Compare commits

...

8 Commits

Author SHA1 Message Date
Adrian Gruntkowski a7603c9e49
Improve import procedure to ensure no time range overlaps (#3970)
* Always scope import ID by site as well

* Do not schedule new import job if there are any site imports in progress

* Disable import buttons when any import is in progress

* Simplify `schedule_job/4` (h/t @RobertJoonas)
3 years ago
Adrian Gruntkowski cffff0340c
Handle Google API timeouts gracefully during imports (#3975) 3 years ago
Adrian Gruntkowski 33eed9d7db
Delete imports which have no stats (#3972) 3 years ago
Marko Saric e5b7f1afd0
Changing the copy of the locked screen (#3967)
changing the copy here as I think that in some situations the "we're still counting stats" message is now shown even to those dashboards where we've stopped counting stats so best to avoid that
3 years ago
Uku Taht f966419a4a
Update ua_inspector (#3957)
Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 f9f0407d68
Remove `experimtnal_hostname_filter` and keep it on by default (#3973)
* Remove `experimental_hostname_filter` and keep it on by default

* Catch up with changes done via e5b56dbe6
3 years ago
RobertJoonas e5b56dbe62
Refactor VisitorGraph (#3936)
* Give a more semantic name to a function

* Make the LineGraph component thinner

* Move LineGraph into a separate file

* Move interval logic into interval-picker.js

This commit also fixes a bug where the interval name displayed inside
the picker component flickers the default interval when the graph is
loading.

The problem was that we were counting on graphData for returning us the
current interval: `let currentInterval = graphData?.interval`

We should always know the default interval before making the main-graph
request. Sending graphData to IntervalPicker component does not make
sense anyway.

* extract data fetching functions out of VisitorGraph component

* Return graph_metric key from Top Stats API

This commit introduces no behavioral changes - only starts returning an
additional field, allowing us to avoid the following logic in React:

1. Finding the metric names, given a stat display name. E.g.
   `Unique visitors (last 30 min) -> visitors`

2. Checking if a metric is graphable or not

* Move metric state into localStorage

This commit gets rid of the internal `metric` state in the VisitorGraph
component and starts using localStorage for that instead.

This commit also chains the main-graph request into the top-stats request
callback - meaning that we'll always fetch new graph data after top stats
are updated. And we do it all in a single function.

Doing so simplifies the loading state significantly, and also helps to
make it clear, that at all times, existing top stats are required before
we can fetch the graph. That's because the metric is determined by which
Top stats are returned (for example, we can't be sure whether revenue
metrics will be returned or not).

* Make sure graph tooltip says "Converted Visitors"

* Extract a StatsExport function component

Again, instead of relying on `graphData?.interval` we can read it from
localStorage, or default to the largest interval available. The export
should not be dependant on the graph.

* Extract SamplingNotice function component

* Extract WithImportedSwitch function component

* Stop "lazy-loading" the graph and top stats

Since the container is always on top on the page, it will be visible on
the first render in any case - no matter the screen size.

* Turn VisitorGraph into a function component

* Display empty container until everything has loaded

* Do not display loading spinner on realtime ticks

* Turn Top Stats into a fn component

* fetch top stats and graph async

* Make sure revenue metrics can remain on the graph

* Add an extra check to canMetricBeGraphed

* fix typo

* remove redundant double negation
3 years ago
Karl-Aksel Puulmann 3115c6e7a8
Reducing JOINs in queries (#3966)
* Move experimental_session_count? logic to within query object

* WIP new querying system for deciding what tables to query

* both -> either

* Include sample_percent in both tables

* Remove a hanging TODO

* Allow filtering by visit props on event queries if flag is on

* Make default sessions join more conditional

* Simplify events_join_sessions?

* Add some TODOs

* Fix assignment

* Handle entry/exit page visit props separately from props stored in events table

* Update test which created sessions/events differently from everyone else

* Make query_events private

* Dont filter by session properties on events table if querying sessions and joining in events

* Handle visits, pageviews, events and visitors metrics from other table

* both -> either

* events, pageviews are strictly event metrics

* Add support for (plain) breakdowns deciding which table to use

* Run tests with experimental_reduced_joins as a separate job

Also refactor which tests are run with postgres:15 to reduce number of jobs

* moduledocs for TableDecider

* Fix matrix

* Custom build name

* Move TEST_EXPERIMENTAL_REDUCED_JOINS check

* Handle percentage separately from other metrics

* Remove debug code

* TableDecider tests

* both => sample_percent

* Improve naming

* Simplify code

* Breakdowns retain old behavior if getting metric visitors

* Unify behavior of entry/exit page hostnames with rest

* Fix test naming
3 years ago

@ -15,14 +15,25 @@ env:
jobs:
build:
name: Build and test
name: "Build and test (${{ matrix.mix_env }}, ${{ matrix.postgres_image }}${{ matrix.test_experimental_reduced_joins == '1' && ', experimental_reduced_joins' || '' }})"
runs-on: ubuntu-latest
strategy:
matrix:
mix_env: ['test', 'small_test']
postgres_image: ['postgres:15', 'postgres:16']
postgres_image: ['postgres:16']
test_experimental_reduced_joins: ['0']
include:
- mix_env: 'test'
postgres_image: 'postgres:15'
test_experimental_reduced_joins: '0'
- mix_env: 'test'
postgres_image: 'postgres:16'
test_experimental_reduced_joins: '1'
env:
MIX_ENV: ${{ matrix.mix_env }}
TEST_EXPERIMENTAL_REDUCED_JOINS: ${{ matrix.test_experimental_reduced_joins }}
services:
postgres:
image: ${{ matrix.postgres_image }}
@ -132,4 +143,4 @@ jobs:
- run: mix format --check-formatted
- run: mix deps.unlock --check-unused
- run: mix credo diff --from-git-merge-base origin/master
- run: mix dialyzer
- run: mix dialyzer

@ -22,6 +22,7 @@ if (container) {
conversionsOptedOut: container.dataset.conversionsOptedOut === 'true',
funnelsOptedOut: container.dataset.funnelsOptedOut === 'true',
propsOptedOut: container.dataset.propsOptedOut === 'true',
revenueGoals: JSON.parse(container.dataset.revenueGoals),
funnels: JSON.parse(container.dataset.funnels),
statsBegin: container.dataset.statsBegin,
nativeStatsBegin: container.dataset.nativeStatsBegin,

@ -44,7 +44,6 @@ export function serializeQuery(query, extraQuery = []) {
if (query.to) { queryObj.to = formatISO(query.to) }
if (query.filters) { queryObj.filters = serializeFilters(query.filters) }
if (query.experimental_session_count) { queryObj.experimental_session_count = query.experimental_session_count }
if (query.experimental_hostname_filter) { queryObj.experimental_hostname_filter = query.experimental_hostname_filter }
if (query.with_imported) { queryObj.with_imported = query.with_imported }
if (SHARED_LINK_AUTH) { queryObj.auth = SHARED_LINK_AUTH }

@ -40,7 +40,6 @@ export function parseQuery(querystring, site) {
match_day_of_week: matchDayOfWeek == 'true',
with_imported: q.get('with_imported') ? q.get('with_imported') === 'true' : true,
experimental_session_count: q.get('experimental_session_count'),
experimental_hostname_filter: q.get('experimental_hostname_filter'),
filters: {
'goal': q.get('goal'),
'props': JSON.parse(q.get('props')),

@ -1,19 +1,41 @@
import numberFormatter, {durationFormatter} from '../../util/number-formatter'
import { parsePrefix } from '../../util/filters'
export const METRIC_MAPPING = {
'Unique visitors (last 30 min)': 'visitors',
'Pageviews (last 30 min)': 'pageviews',
'Unique visitors': 'visitors',
'Visit duration': 'visit_duration',
'Total pageviews': 'pageviews',
'Views per visit': 'views_per_visit',
'Total visits': 'visits',
'Bounce rate': 'bounce_rate',
'Unique conversions': 'conversions',
'Total conversions': 'events',
'Conversion rate': 'conversion_rate',
'Average revenue': 'average_revenue',
'Total revenue': 'total_revenue',
export function getGraphableMetrics(query, site) {
const isRealtime = query.period === 'realtime'
const goalFilter = query.filters.goal
const pageFilter = query.filters.page
if (isRealtime && goalFilter) {
return ["visitors"]
} else if (isRealtime) {
return ["visitors", "pageviews"]
} else if (goalFilter && canGraphRevenueMetrics(goalFilter, site)) {
return ["visitors", "events", "average_revenue", "total_revenue", "conversion_rate"]
} else if (goalFilter) {
return ["visitors", "events", "conversion_rate"]
} else if (pageFilter) {
return ["visitors", "visits", "pageviews", "bounce_rate", "time_on_page"]
} else {
return ["visitors", "visits", "pageviews", "views_per_visit", "bounce_rate", "visit_duration"]
}
}
// Revenue metrics can only be graphed if:
// * The query is filtered by at least one revenue goal
// * All revenue goals in filter have the same currency
function canGraphRevenueMetrics(goalFilter, site) {
const goalsInFilter = parsePrefix(goalFilter).values
const revenueGoalsInFilter = site.revenueGoals.filter((rg) => {
return goalsInFilter.includes(rg.event_name)
})
const singleCurrency = revenueGoalsInFilter.every((rg) => {
return rg.currency === revenueGoalsInFilter[0].currency
})
return revenueGoalsInFilter.length > 0 && singleCurrency
}
export const METRIC_LABELS = {
@ -44,14 +66,6 @@ export const METRIC_FORMATTER = {
'average_revenue': numberFormatter,
}
export const LoadingState = {
loading: 'loading',
refreshing: 'refreshing',
loaded: 'loaded',
isLoadingOrRefreshing: function (state) { return [this.loading, this.refreshing].includes(state) },
isLoadedOrRefreshing: function (state) { return [this.loaded, this.refreshing].includes(state) }
}
const buildComparisonDataset = function(comparisonPlot) {
if (!comparisonPlot) return []

@ -6,7 +6,7 @@ import * as storage from '../../util/storage'
import { isKeyPressed } from '../../keybinding.js'
import { monthsBetweenDates } from '../../util/date.js'
export const INTERVAL_LABELS = {
const INTERVAL_LABELS = {
'minute': 'Minutes',
'hour': 'Hours',
'date': 'Days',
@ -14,11 +14,19 @@ export const INTERVAL_LABELS = {
'month': 'Months'
}
export const getStoredInterval = function(period, domain) {
function validIntervals(site, query) {
if (query.period === "custom" && monthsBetweenDates(query.from, query.to) > 12) {
return ["week", "month"]
} else {
return site.validIntervalsByPeriod[query.period]
}
}
function getStoredInterval(period, domain) {
return storage.getItem(`interval__${period}__${domain}`)
}
export const storeInterval = function(period, domain, interval) {
function storeInterval(period, domain, interval) {
storage.setItem(`interval__${period}__${domain}`, interval)
}
@ -33,37 +41,47 @@ function subscribeKeybinding(element) {
}, [handleKeyPress])
}
function DropdownItem({ option, currentInterval, updateInterval }) {
return (
<Menu.Item onClick={() => updateInterval(option)} key={option} disabled={option == currentInterval}>
{({ active }) => (
<span className={classNames({
'bg-gray-100 dark:bg-gray-900 text-gray-900 dark:text-gray-200 cursor-pointer': active,
'text-gray-700 dark:text-gray-200': !active,
'font-bold cursor-none select-none': option == currentInterval,
}, 'block px-4 py-2 text-sm')}>
{INTERVAL_LABELS[option]}
</span>
)}
</Menu.Item>
)
export const getCurrentInterval = function(site, query) {
const options = validIntervals(site, query)
const storedInterval = getStoredInterval(query.period, site.domain)
const defaultInterval = [...options].pop()
if (storedInterval && options.includes(storedInterval)) {
return storedInterval
} else {
return defaultInterval
}
}
export function IntervalPicker({ graphData, query, site, updateInterval }) {
export function IntervalPicker({ query, site, onIntervalUpdate }) {
if (query.period == 'realtime') return null
const menuElement = React.useRef(null)
subscribeKeybinding(menuElement)
const options = validIntervals(site, query)
const currentInterval = getCurrentInterval(site, query)
let currentInterval = graphData?.interval
subscribeKeybinding(menuElement)
let options = site.validIntervalsByPeriod[query.period]
if (query.period === "custom" && monthsBetweenDates(query.from, query.to) > 12) {
options = ["week", "month"]
function updateInterval(interval) {
storeInterval(query.period, site.domain, interval)
onIntervalUpdate(interval)
}
if (!options.includes(currentInterval)) {
currentInterval = [...options].pop()
function renderDropdownItem(option) {
return (
<Menu.Item onClick={() => updateInterval(option)} key={option} disabled={option == currentInterval}>
{({ active }) => (
<span className={classNames({
'bg-gray-100 dark:bg-gray-900 text-gray-900 dark:text-gray-200 cursor-pointer': active,
'text-gray-700 dark:text-gray-200': !active,
'font-bold cursor-none select-none': option == currentInterval,
}, 'block px-4 py-2 text-sm')}>
{INTERVAL_LABELS[option]}
</span>
)}
</Menu.Item>
)
}
return (
@ -85,7 +103,7 @@ export function IntervalPicker({ graphData, query, site, updateInterval }) {
leaveFrom="opacity-100 scale-100"
leaveTo="opacity-0 scale-95">
<Menu.Items className="py-1 text-left origin-top-right absolute right-0 mt-2 w-56 rounded-md shadow-lg bg-white dark:bg-gray-800 ring-1 ring-black ring-opacity-5 focus:outline-none z-10" static>
{options.map((option) => DropdownItem({ option, currentInterval, updateInterval }))}
{options.map(renderDropdownItem)}
</Menu.Items>
</Transition>
</>

@ -0,0 +1,247 @@
import React from 'react';
import { withRouter } from 'react-router-dom'
import Chart from 'chart.js/auto';
import { navigateToQuery } from '../../query'
import GraphTooltip from './graph-tooltip'
import { buildDataSet, METRIC_LABELS, METRIC_FORMATTER } from './graph-util'
import dateFormatter from './date-formatter';
import FadeIn from '../../fade-in';
import classNames from 'classnames';
const calculateMaximumY = function(dataset) {
const yAxisValues = dataset
.flatMap((item) => item.data)
.map((item) => item || 0)
if (yAxisValues) {
return Math.max(...yAxisValues)
} else {
return 1
}
}
class LineGraph extends React.Component {
constructor(props) {
super(props);
this.regenerateChart = this.regenerateChart.bind(this);
this.updateWindowDimensions = this.updateWindowDimensions.bind(this);
}
getGraphMetric() {
let metric = this.props.graphData.metric
if (metric == 'visitors' && this.props.query.filters.goal) {
return 'conversions'
} else {
return metric
}
}
regenerateChart() {
const { graphData, query } = this.props
const metric = this.getGraphMetric()
const graphEl = document.getElementById("main-graph-canvas")
this.ctx = graphEl.getContext('2d');
const dataSet = buildDataSet(graphData.plot, graphData.comparison_plot, graphData.present_index, this.ctx, METRIC_LABELS[metric])
return new Chart(this.ctx, {
type: 'line',
data: {
labels: graphData.labels,
datasets: dataSet
},
options: {
animation: false,
plugins: {
legend: { display: false },
tooltip: {
enabled: false,
mode: 'index',
intersect: false,
position: 'average',
external: GraphTooltip(graphData, metric, query)
},
},
responsive: true,
maintainAspectRatio: false,
onResize: this.updateWindowDimensions,
elements: { line: { tension: 0 }, point: { radius: 0 } },
onClick: this.maybeHopToHoveredPeriod.bind(this),
scale: {
ticks: { precision: 0, maxTicksLimit: 8 }
},
scales: {
y: {
min: 0,
suggestedMax: calculateMaximumY(dataSet),
ticks: {
callback: METRIC_FORMATTER[metric],
color: this.props.darkTheme ? 'rgb(243, 244, 246)' : undefined
},
grid: {
zeroLineColor: 'transparent',
drawBorder: false,
}
},
yComparison: {
min: 0,
suggestedMax: calculateMaximumY(dataSet),
display: false,
grid: { display: false },
},
x: {
grid: { display: false },
ticks: {
callback: function(val, _index, _ticks) {
if (this.getLabelForValue(val) == "__blank__") return ""
const hasMultipleYears =
graphData.labels
.filter((date) => typeof date === 'string')
.map(date => date.split('-')[0])
.filter((value, index, list) => list.indexOf(value) === index)
.length > 1
if (graphData.interval === 'hour' && query.period !== 'day') {
const date = dateFormatter({
interval: "date",
longForm: false,
period: query.period,
shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
const hour = dateFormatter({
interval: graphData.interval,
longForm: false,
period: query.period,
shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
// Returns a combination of date and hour. This is because
// small intervals like hour may return multiple days
// depending on the query period.
return `${date}, ${hour}`
}
if (graphData.interval === 'minute' && query.period !== 'realtime') {
return dateFormatter({
interval: "hour", longForm: false, period: query.period,
})(this.getLabelForValue(val))
}
return dateFormatter({
interval: graphData.interval, longForm: false, period: query.period, shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
},
color: this.props.darkTheme ? 'rgb(243, 244, 246)' : undefined
}
}
},
interaction: {
mode: 'index',
intersect: false,
}
}
});
}
repositionTooltip(e) {
const tooltipEl = document.getElementById('chartjs-tooltip');
if (tooltipEl && window.innerWidth >= 768) {
if (e.clientX > 0.66 * window.innerWidth) {
tooltipEl.style.right = (window.innerWidth - e.clientX) + window.pageXOffset + 'px'
tooltipEl.style.left = null;
} else {
tooltipEl.style.right = null;
tooltipEl.style.left = e.clientX + window.pageXOffset + 'px'
}
tooltipEl.style.top = e.clientY + window.pageYOffset + 'px'
tooltipEl.style.opacity = 1;
}
}
componentDidMount() {
if (this.props.graphData) {
this.chart = this.regenerateChart();
}
window.addEventListener('mousemove', this.repositionTooltip);
}
componentDidUpdate(prevProps) {
const { graphData, darkTheme } = this.props;
const tooltip = document.getElementById('chartjs-tooltip');
if (
graphData !== prevProps.graphData ||
darkTheme !== prevProps.darkTheme
) {
if (graphData) {
if (this.chart) {
this.chart.destroy();
}
this.chart = this.regenerateChart();
this.chart.update();
}
if (tooltip) {
tooltip.style.display = 'none';
}
}
if (!graphData) {
if (this.chart) {
this.chart.destroy();
}
if (tooltip) {
tooltip.style.display = 'none';
}
}
}
componentWillUnmount() {
// Ensure that the tooltip doesn't hang around when we are loading more data
const tooltip = document.getElementById('chartjs-tooltip');
if (tooltip) {
tooltip.style.opacity = 0;
tooltip.style.display = 'none';
}
window.removeEventListener('mousemove', this.repositionTooltip)
}
/**
* The current ticks' limits are set to treat iPad (regular/Mini/Pro) as a regular screen.
* @param {*} chart - The chart instance.
* @param {*} dimensions - An object containing the new dimensions *of the chart.*
*/
updateWindowDimensions(chart, dimensions) {
chart.options.scales.x.ticks.maxTicksLimit = dimensions.width < 720 ? 5 : 8
}
maybeHopToHoveredPeriod(e) {
const element = this.chart.getElementsAtEventForMode(e, 'index', { intersect: false })[0]
const date = this.props.graphData.labels[element.index] || this.props.graphData.comparison_labels[element.index]
if (this.props.graphData.interval === 'month') {
navigateToQuery(this.props.history, this.props.query, { period: 'month', date })
} else if (this.props.graphData.interval === 'date') {
navigateToQuery(this.props.history, this.props.query, { period: 'day', date })
}
}
render() {
const { graphData } = this.props
const canvasClass = classNames('mt-4 select-none', { 'cursor-pointer': !['minute', 'hour'].includes(graphData?.interval) })
return (
<FadeIn show={graphData}>
<div className="relative h-96 print:h-auto print:pb-8 w-full z-0">
<canvas id="main-graph-canvas" className={canvasClass}></canvas>
</div>
</FadeIn>
)
}
}
export default withRouter(LineGraph)

@ -0,0 +1,17 @@
import React from "react"
export default function SamplingNotice({topStatData}) {
const samplePercent = topStatData?.samplePercent
if (samplePercent && samplePercent < 100) {
return (
<div tooltip={`Stats based on a ${samplePercent}% sample of all visitors`} className="cursor-pointer w-4 h-4 mx-2">
<svg className="absolute w-4 h-4 dark:text-gray-300 text-gray-700" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
</div>
)
} else {
return null
}
}

@ -0,0 +1,53 @@
import React, { useState } from "react"
import * as api from '../../api'
import { getCurrentInterval } from "./interval-picker"
export default function StatsExport({site, query}) {
const [exporting, setExporting] = useState(false)
function startExport() {
setExporting(true)
document.cookie = "exporting="
pollExportReady()
}
function pollExportReady() {
if (document.cookie.includes('exporting')) {
setTimeout(pollExportReady, 1000)
} else {
setExporting(false)
}
}
function renderLoading() {
return (
<svg className="animate-spin h-4 w-4 text-indigo-500" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24">
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4"></circle>
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
</svg>
)
}
function renderExportLink() {
const interval = getCurrentInterval(site, query)
const queryParams = api.serializeQuery(query, [{ interval }])
const endpoint = `/${encodeURIComponent(site.domain)}/export${queryParams}`
return (
<a href={endpoint} download onClick={startExport}>
<svg className="absolute text-gray-700 feather dark:text-gray-300 hover:text-indigo-600 dark:hover:text-indigo-600" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"></path>
<polyline points="7 10 12 15 17 10"></polyline>
<line x1="12" y1="15" x2="12" y2="3"></line>
</svg>
</a>
)
}
return (
<div className="w-4 h-4 mx-2">
{exporting && renderLoading()}
{!exporting && renderExportLink()}
</div>
)
}

@ -3,8 +3,9 @@ import { Tooltip } from '../../util/tooltip'
import { SecondsSinceLastLoad } from '../../util/seconds-since-last-load'
import classNames from "classnames";
import numberFormatter, { durationFormatter } from '../../util/number-formatter'
import { METRIC_MAPPING } from './graph-util'
import * as storage from '../../util/storage'
import { formatDateRange } from '../../util/date.js'
import { getGraphableMetrics } from "./graph-util.js";
function Maybe({condition, children}) {
if (condition) {
@ -14,99 +15,103 @@ function Maybe({condition, children}) {
}
}
export default class TopStats extends React.Component {
renderPercentageComparison(name, comparison, forceDarkBg = false) {
const formattedComparison = numberFormatter(Math.abs(comparison))
const defaultClassName = classNames({
"pl-2 text-xs dark:text-gray-100": !forceDarkBg,
"pl-2 text-xs text-gray-100": forceDarkBg
})
const noChangeClassName = classNames({
"pl-2 text-xs text-gray-700 dark:text-gray-300": !forceDarkBg,
"pl-2 text-xs text-gray-300": forceDarkBg
})
if (comparison > 0) {
const color = name === 'Bounce rate' ? 'text-red-400' : 'text-green-500'
return <span className={defaultClassName}><span className={color + ' font-bold'}>&uarr;</span> {formattedComparison}%</span>
} else if (comparison < 0) {
const color = name === 'Bounce rate' ? 'text-green-500' : 'text-red-400'
return <span className={defaultClassName}><span className={color + ' font-bold'}>&darr;</span> {formattedComparison}%</span>
} else if (comparison === 0) {
return <span className={noChangeClassName}>&#12336; 0%</span>
} else {
return null
}
function renderPercentageComparison(name, comparison, forceDarkBg = false) {
const formattedComparison = numberFormatter(Math.abs(comparison))
const defaultClassName = classNames({
"pl-2 text-xs dark:text-gray-100": !forceDarkBg,
"pl-2 text-xs text-gray-100": forceDarkBg
})
const noChangeClassName = classNames({
"pl-2 text-xs text-gray-700 dark:text-gray-300": !forceDarkBg,
"pl-2 text-xs text-gray-300": forceDarkBg
})
if (comparison > 0) {
const color = name === 'Bounce rate' ? 'text-red-400' : 'text-green-500'
return <span className={defaultClassName}><span className={color + ' font-bold'}>&uarr;</span> {formattedComparison}%</span>
} else if (comparison < 0) {
const color = name === 'Bounce rate' ? 'text-green-500' : 'text-red-400'
return <span className={defaultClassName}><span className={color + ' font-bold'}>&darr;</span> {formattedComparison}%</span>
} else if (comparison === 0) {
return <span className={noChangeClassName}>&#12336; 0%</span>
} else {
return null
}
}
topStatNumberShort(name, value) {
if (['visit duration', 'time on page'].includes(name.toLowerCase())) {
return durationFormatter(value)
} else if (['bounce rate', 'conversion rate'].includes(name.toLowerCase())) {
return value + '%'
} else if (['average revenue', 'total revenue'].includes(name.toLowerCase())) {
return value?.short
} else {
return numberFormatter(value)
}
function topStatNumberShort(name, value) {
if (['visit duration', 'time on page'].includes(name.toLowerCase())) {
return durationFormatter(value)
} else if (['bounce rate', 'conversion rate'].includes(name.toLowerCase())) {
return value + '%'
} else if (['average revenue', 'total revenue'].includes(name.toLowerCase())) {
return value?.short
} else {
return numberFormatter(value)
}
}
topStatNumberLong(name, value) {
if (['visit duration', 'time on page'].includes(name.toLowerCase())) {
return durationFormatter(value)
} else if (['bounce rate', 'conversion rate'].includes(name.toLowerCase())) {
return value + '%'
} else if (['average revenue', 'total revenue'].includes(name.toLowerCase())) {
return value?.long
} else {
return (value || 0).toLocaleString()
}
function topStatNumberLong(name, value) {
if (['visit duration', 'time on page'].includes(name.toLowerCase())) {
return durationFormatter(value)
} else if (['bounce rate', 'conversion rate'].includes(name.toLowerCase())) {
return value + '%'
} else if (['average revenue', 'total revenue'].includes(name.toLowerCase())) {
return value?.long
} else {
return (value || 0).toLocaleString()
}
}
export default function TopStats(props) {
const {site, query, data, onMetricUpdate, tooltipBoundary, lastLoadTimestamp} = props
topStatTooltip(stat, query) {
function tooltip(stat) {
let statName = stat.name.toLowerCase()
statName = stat.value === 1 ? statName.slice(0, -1) : statName
return (
<div>
{query.comparison && <div className="whitespace-nowrap">
{this.topStatNumberLong(stat.name, stat.value)} vs. {this.topStatNumberLong(stat.name, stat.comparison_value)} {statName}
<span className="ml-2">{this.renderPercentageComparison(stat.name, stat.change, true)}</span>
{topStatNumberLong(stat.name, stat.value)} vs. {topStatNumberLong(stat.name, stat.comparison_value)} {statName}
<span className="ml-2">{renderPercentageComparison(stat.name, stat.change, true)}</span>
</div>}
{!query.comparison && <div className="whitespace-nowrap">
{this.topStatNumberLong(stat.name, stat.value)} {statName}
{topStatNumberLong(stat.name, stat.value)} {statName}
</div>}
{stat.name === 'Current visitors' && <p className="font-normal text-xs">Last updated <SecondsSinceLastLoad lastLoadTimestamp={this.props.lastLoadTimestamp}/>s ago</p>}
{stat.name === 'Current visitors' && <p className="font-normal text-xs">Last updated <SecondsSinceLastLoad lastLoadTimestamp={lastLoadTimestamp}/>s ago</p>}
</div>
)
}
canMetricBeGraphed(stat) {
const isTotalUniqueVisitors = this.props.query.filters.goal && stat.name === 'Unique visitors'
const isKnownMetric = Object.keys(METRIC_MAPPING).includes(stat.name)
return isKnownMetric && !isTotalUniqueVisitors
function canMetricBeGraphed(stat) {
const graphableMetrics = getGraphableMetrics(query, site)
return stat.graph_metric && graphableMetrics.includes(stat.graph_metric)
}
maybeUpdateMetric(stat) {
if (this.canMetricBeGraphed(stat)) {
this.props.updateMetric(METRIC_MAPPING[stat.name])
function maybeUpdateMetric(stat) {
if (canMetricBeGraphed(stat)) {
storage.setItem(`metric__${site.domain}`, stat.graph_metric)
onMetricUpdate(stat.graph_metric)
}
}
blinkingDot() {
function blinkingDot() {
return (
<div key="dot" className="block pulsating-circle" style={{ left: '125px', top: '52px' }}></div>
)
}
renderStatName(stat) {
const { metric } = this.props
const isSelected = metric === METRIC_MAPPING[stat.name]
function getStoredMetric() {
return storage.getItem(`metric__${site.domain}`)
}
function renderStatName(stat) {
const isSelected = stat.graph_metric === getStoredMetric()
const [statDisplayName, statExtraName] = stat.name.split(/(\(.+\))/g)
@ -123,48 +128,45 @@ export default class TopStats extends React.Component {
)
}
render() {
const { topStatData, query, site } = this.props
const stats = topStatData && topStatData.top_stats.map((stat, index) => {
const className = classNames('px-4 md:px-6 w-1/2 my-4 lg:w-auto group select-none', {
'cursor-pointer': this.canMetricBeGraphed(stat),
'lg:border-l border-gray-300': index > 0,
'border-r lg:border-r-0': index % 2 === 0
})
return (
<Tooltip key={stat.name} info={this.topStatTooltip(stat, query)} className={className} onClick={() => { this.maybeUpdateMetric(stat) }} boundary={this.props.tooltipBoundary}>
{this.renderStatName(stat)}
<div className="my-1 space-y-2">
<div>
<span className="flex items-center justify-between whitespace-nowrap">
<p className="font-bold text-xl dark:text-gray-100" id={METRIC_MAPPING[stat.name]}>{this.topStatNumberShort(stat.name, stat.value)}</p>
<Maybe condition={!query.comparison}>
{ this.renderPercentageComparison(stat.name, stat.change) }
</Maybe>
</span>
<Maybe condition={query.comparison}>
<p className="text-xs dark:text-gray-100">{ formatDateRange(site, topStatData.from, topStatData.to) }</p>
</Maybe>
</div>
function renderStat(stat, index) {
const className = classNames('px-4 md:px-6 w-1/2 my-4 lg:w-auto group select-none', {
'cursor-pointer': canMetricBeGraphed(stat),
'lg:border-l border-gray-300': index > 0,
'border-r lg:border-r-0': index % 2 === 0
})
return (
<Tooltip key={stat.name} info={tooltip(stat, query)} className={className} onClick={() => { maybeUpdateMetric(stat) }} boundary={tooltipBoundary}>
{renderStatName(stat)}
<div className="my-1 space-y-2">
<div>
<span className="flex items-center justify-between whitespace-nowrap">
<p className="font-bold text-xl dark:text-gray-100" id={stat.graph_metric}>{topStatNumberShort(stat.name, stat.value)}</p>
<Maybe condition={!query.comparison}>
{ renderPercentageComparison(stat.name, stat.change) }
</Maybe>
</span>
<Maybe condition={query.comparison}>
<div>
<p className="font-bold text-xl text-gray-500 dark:text-gray-400">{ this.topStatNumberShort(stat.name, stat.comparison_value) }</p>
<p className="text-xs text-gray-500 dark:text-gray-400">{ formatDateRange(site, topStatData.comparing_from, topStatData.comparing_to) }</p>
</div>
<p className="text-xs dark:text-gray-100">{ formatDateRange(site, data.from, data.to) }</p>
</Maybe>
</div>
<Maybe condition={query.comparison}>
<div>
<p className="font-bold text-xl text-gray-500 dark:text-gray-400">{ topStatNumberShort(stat.name, stat.comparison_value) }</p>
<p className="text-xs text-gray-500 dark:text-gray-400">{ formatDateRange(site, data.comparing_from, data.comparing_to) }</p>
</div>
</Tooltip>
)
})
</Maybe>
</div>
</Tooltip>
)
}
if (stats && query && query.period === 'realtime') {
stats.push(this.blinkingDot())
}
const stats = data && data.top_stats.map(renderStat)
return stats || null;
if (stats && query.period === 'realtime') {
stats.push(blinkingDot())
}
return stats || null;
}

@ -1,532 +1,153 @@
import React from 'react';
import { withRouter, Link } from 'react-router-dom'
import Chart from 'chart.js/auto';
import { navigateToQuery } from '../../query'
import React, { useState, useEffect, useRef, useCallback } from 'react';
import * as api from '../../api'
import * as storage from '../../util/storage'
import LazyLoader from '../../components/lazy-loader'
import GraphTooltip from './graph-tooltip'
import { buildDataSet, METRIC_MAPPING, METRIC_LABELS, METRIC_FORMATTER, LoadingState } from './graph-util'
import dateFormatter from './date-formatter';
import { getGraphableMetrics } from './graph-util'
import TopStats from './top-stats';
import { IntervalPicker, getStoredInterval, storeInterval } from './interval-picker';
import { IntervalPicker, getCurrentInterval } from './interval-picker'
import StatsExport from './stats-export'
import WithImportedSwitch from './with-imported-switch';
import SamplingNotice from './sampling-notice';
import FadeIn from '../../fade-in';
import * as url from '../../util/url'
import classNames from 'classnames';
import { monthsBetweenDates, parseNaiveDate, isBefore } from '../../util/date'
import { isComparisonEnabled } from '../../comparison-input'
import { BarsArrowUpIcon } from '@heroicons/react/20/solid'
import LineGraphWithRouter from './line-graph'
const calculateMaximumY = function(dataset) {
const yAxisValues = dataset
.flatMap((item) => item.data)
.map((item) => item || 0)
if (yAxisValues) {
return Math.max(...yAxisValues)
} else {
return 1
}
}
class LineGraph extends React.Component {
constructor(props) {
super(props);
this.boundary = React.createRef()
this.regenerateChart = this.regenerateChart.bind(this);
this.updateWindowDimensions = this.updateWindowDimensions.bind(this);
this.state = {
exported: false
};
function fetchTopStats(site, query) {
const q = { ...query }
if (!isComparisonEnabled(q.comparison)) {
q.comparison = 'previous_period'
}
regenerateChart() {
const { graphData, metric, query } = this.props
const graphEl = document.getElementById("main-graph-canvas")
this.ctx = graphEl.getContext('2d');
const dataSet = buildDataSet(graphData.plot, graphData.comparison_plot, graphData.present_index, this.ctx, METRIC_LABELS[metric])
return new Chart(this.ctx, {
type: 'line',
data: {
labels: graphData.labels,
datasets: dataSet
},
options: {
animation: false,
plugins: {
legend: { display: false },
tooltip: {
enabled: false,
mode: 'index',
intersect: false,
position: 'average',
external: GraphTooltip(graphData, metric, query)
},
},
responsive: true,
maintainAspectRatio: false,
onResize: this.updateWindowDimensions,
elements: { line: { tension: 0 }, point: { radius: 0 } },
onClick: this.onClick.bind(this),
scale: {
ticks: { precision: 0, maxTicksLimit: 8 }
},
scales: {
y: {
min: 0,
suggestedMax: calculateMaximumY(dataSet),
ticks: {
callback: METRIC_FORMATTER[metric],
color: this.props.darkTheme ? 'rgb(243, 244, 246)' : undefined
},
grid: {
zeroLineColor: 'transparent',
drawBorder: false,
}
},
yComparison: {
min: 0,
suggestedMax: calculateMaximumY(dataSet),
display: false,
grid: { display: false },
},
x: {
grid: { display: false },
ticks: {
callback: function(val, _index, _ticks) {
if (this.getLabelForValue(val) == "__blank__") return ""
const hasMultipleYears =
graphData.labels
.filter((date) => typeof date === 'string')
.map(date => date.split('-')[0])
.filter((value, index, list) => list.indexOf(value) === index)
.length > 1
if (graphData.interval === 'hour' && query.period !== 'day') {
const date = dateFormatter({
interval: "date",
longForm: false,
period: query.period,
shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
const hour = dateFormatter({
interval: graphData.interval,
longForm: false,
period: query.period,
shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
return api.get(url.apiPath(site, '/top-stats'), q)
}
// Returns a combination of date and hour. This is because
// small intervals like hour may return multiple days
// depending on the query period.
return `${date}, ${hour}`
}
function fetchMainGraph(site, query, metric, interval) {
const params = {metric, interval}
return api.get(url.apiPath(site, '/main-graph'), query, params)
}
if (graphData.interval === 'minute' && query.period !== 'realtime') {
return dateFormatter({
interval: "hour", longForm: false, period: query.period,
})(this.getLabelForValue(val))
}
export default function VisitorGraph(props) {
const {site, query, lastLoadTimestamp} = props
const isRealtime = query.period === 'realtime'
const isDarkTheme = document.querySelector('html').classList.contains('dark') || false
return dateFormatter({
interval: graphData.interval, longForm: false, period: query.period, shouldShowYear: hasMultipleYears,
})(this.getLabelForValue(val))
},
color: this.props.darkTheme ? 'rgb(243, 244, 246)' : undefined
}
}
},
interaction: {
mode: 'index',
intersect: false,
}
}
});
}
repositionTooltip(e) {
const tooltipEl = document.getElementById('chartjs-tooltip');
if (tooltipEl && window.innerWidth >= 768) {
if (e.clientX > 0.66 * window.innerWidth) {
tooltipEl.style.right = (window.innerWidth - e.clientX) + window.pageXOffset + 'px'
tooltipEl.style.left = null;
} else {
tooltipEl.style.right = null;
tooltipEl.style.left = e.clientX + window.pageXOffset + 'px'
}
tooltipEl.style.top = e.clientY + window.pageYOffset + 'px'
tooltipEl.style.opacity = 1;
}
}
const topStatsBoundary = useRef(null)
componentDidMount() {
if (this.props.graphData) {
this.chart = this.regenerateChart();
}
window.addEventListener('mousemove', this.repositionTooltip);
}
const [topStatData, setTopStatData] = useState(null)
const [topStatsLoading, setTopStatsLoading] = useState(true)
const [graphData, setGraphData] = useState(null)
const [graphLoading, setGraphLoading] = useState(true)
componentDidUpdate(prevProps) {
const { graphData, darkTheme } = this.props;
const tooltip = document.getElementById('chartjs-tooltip');
// This state is explicitly meant for the situation where either graph interval
// or graph metric is changed. That results in behaviour where Top Stats stay
// intact, but the graph container alone will display a loading spinner for as
// long as new graph data is fetched.
const [graphRefreshing, setGraphRefreshing] = useState(false)
if (
graphData !== prevProps.graphData ||
darkTheme !== prevProps.darkTheme
) {
if (graphData) {
if (this.chart) {
this.chart.destroy();
}
this.chart = this.regenerateChart();
this.chart.update();
}
const onIntervalUpdate = useCallback((newInterval) => {
setGraphData(null)
setGraphRefreshing(true)
fetchGraphData(getStoredMetric(), newInterval)
}, [query])
if (tooltip) {
tooltip.style.display = 'none';
}
}
const onMetricUpdate = useCallback((newMetric) => {
setGraphData(null)
setGraphRefreshing(true)
fetchGraphData(newMetric, getCurrentInterval(site, query))
}, [query])
if (!graphData) {
if (this.chart) {
this.chart.destroy();
}
useEffect(() => {
setTopStatData(null)
setTopStatsLoading(true)
setGraphData(null)
setGraphLoading(true)
fetchTopStatsAndGraphData()
if (tooltip) {
tooltip.style.display = 'none';
}
if (isRealtime) {
document.addEventListener('tick', fetchTopStatsAndGraphData)
}
}
componentWillUnmount() {
// Ensure that the tooltip doesn't hang around when we are loading more data
const tooltip = document.getElementById('chartjs-tooltip');
if (tooltip) {
tooltip.style.opacity = 0;
tooltip.style.display = 'none';
return () => {
document.removeEventListener('tick', fetchTopStatsAndGraphData)
}
window.removeEventListener('mousemove', this.repositionTooltip)
}
/**
* The current ticks' limits are set to treat iPad (regular/Mini/Pro) as a regular screen.
* @param {*} chart - The chart instance.
* @param {*} dimensions - An object containing the new dimensions *of the chart.*
*/
updateWindowDimensions(chart, dimensions) {
chart.options.scales.x.ticks.maxTicksLimit = dimensions.width < 720 ? 5 : 8
}
}, [query])
onClick(e) {
const element = this.chart.getElementsAtEventForMode(e, 'index', { intersect: false })[0]
const date = this.props.graphData.labels[element.index] || this.props.graphData.comparison_labels[element.index]
useEffect(() => {
if (topStatData) { storeTopStatsContainerHeight() }
}, [topStatData])
if (this.props.graphData.interval === 'month') {
navigateToQuery(this.props.history, this.props.query, { period: 'month', date })
} else if (this.props.graphData.interval === 'date') {
navigateToQuery(this.props.history, this.props.query, { period: 'day', date })
function fetchTopStatsAndGraphData() {
fetchTopStats(site, query)
.then((res) => {
setTopStatData(res)
setTopStatsLoading(false)
})
let metric = getStoredMetric()
const availableMetrics = getGraphableMetrics(query, site)
if (!availableMetrics.includes(metric)) {
metric = availableMetrics[0]
storage.setItem(`metric__${site.domain}`, metric)
}
}
pollExportReady() {
if (document.cookie.includes('exporting')) {
setTimeout(this.pollExportReady.bind(this), 1000);
} else {
this.setState({ exported: false })
}
}
const interval = getCurrentInterval(site, query)
downloadSpinner() {
this.setState({ exported: true });
document.cookie = "exporting=";
setTimeout(this.pollExportReady.bind(this), 1000);
fetchGraphData(metric, interval)
}
downloadLink() {
if (this.props.query.period !== 'realtime') {
if (this.state.exported) {
return (
<div className="w-4 h-4 mx-2">
<svg className="animate-spin h-4 w-4 text-indigo-500" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24">
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4"></circle>
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
</svg>
</div>
)
} else {
const interval = this.props.graphData?.interval
const queryParams = api.serializeQuery(this.props.query, [{ interval }])
const endpoint = `/${encodeURIComponent(this.props.site.domain)}/export${queryParams}`
return (
<a className="w-4 h-4 mx-2" href={endpoint} download onClick={this.downloadSpinner.bind(this)}>
<svg className="absolute text-gray-700 feather dark:text-gray-300 hover:text-indigo-600 dark:hover:text-indigo-600" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"></path><polyline points="7 10 12 15 17 10"></polyline><line x1="12" y1="15" x2="12" y2="3"></line></svg>
</a>
)
}
}
function fetchGraphData(metric, interval) {
fetchMainGraph(site, query, metric, interval)
.then((res) => {
setGraphData(res)
setGraphLoading(false)
setGraphRefreshing(false)
})
}
samplingNotice() {
const samplePercent = this.props.topStatData && this.props.topStatData.sample_percent
if (samplePercent < 100) {
return (
<div tooltip={`Stats based on a ${samplePercent}% sample of all visitors`} className="cursor-pointer w-4 h-4 mx-2">
<svg className="absolute w-4 h-4 dark:text-gray-300 text-gray-700" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth="2" d="M13 16h-1v-4h-1m1-4h.01M21 12a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
</div>
)
}
function getStoredMetric() {
return storage.getItem(`metric__${site.domain}`)
}
importedNotice() {
if (!this.props.topStatData?.imports_exist) return
const isBeforeNativeStats = (date) => {
if (!date) return false
const nativeStatsBegin = parseNaiveDate(this.props.site.nativeStatsBegin)
const parsedDate = parseNaiveDate(date)
return isBefore(parsedDate, nativeStatsBegin, "day")
}
const isQueryingImportedPeriod = isBeforeNativeStats(this.props.topStatData.from)
const isComparingImportedPeriod = isBeforeNativeStats(this.props.topStatData.comparing_from)
if (isQueryingImportedPeriod || isComparingImportedPeriod) {
const withImported = this.props.topStatData.with_imported;
const toggleColor = withImported ? " dark:text-gray-300 text-gray-700" : " dark:text-gray-500 text-gray-400"
const target = url.setQuery('with_imported', !withImported)
const tip = withImported ? "" : "do not ";
return (
<Link to={target} className="w-4 h-4 mx-2">
<div tooltip={`Stats ${tip}include imported data.`} className="cursor-pointer w-4 h-4">
<BarsArrowUpIcon className={"absolute " + toggleColor} />
</div>
</Link>
)
}
function storeTopStatsContainerHeight() {
storage.setItem(`topStatsHeight__${site.domain}`, document.getElementById('top-stats-container').clientHeight)
}
// This function is used for maintaining the main-graph/top-stats container height in the
// loading process. The container height depends on how many top stat metrics are returned
// from the API, but in the loading state, we don't know that yet. We can use localStorage
// to keep track of the Top Stats container height.
getTopStatsHeight() {
if (this.props.topStatData) {
function getTopStatsHeight() {
if (topStatData) {
return 'auto'
} else {
return `${storage.getItem(`topStatsHeight__${this.props.site.domain}`) || 89}px`
return `${storage.getItem(`topStatsHeight__${site.domain}`) || 89}px`
}
}
render() {
const { mainGraphRefreshing, updateMetric, updateInterval, metric, topStatData, query, site, graphData, lastLoadTimestamp } = this.props
const canvasClass = classNames('mt-4 select-none', { 'cursor-pointer': !['minute', 'hour'].includes(graphData?.interval) })
return (
<div>
<div id="top-stats-container" className="flex flex-wrap" ref={this.boundary} style={{ height: this.getTopStatsHeight() }}>
<TopStats site={site} query={query} metric={metric} updateMetric={updateMetric} topStatData={topStatData} tooltipBoundary={this.boundary.current} lastLoadTimestamp={lastLoadTimestamp} />
return (
<div className={"relative w-full mt-2 bg-white rounded shadow-xl dark:bg-gray-825"}>
{(topStatsLoading || graphLoading) && renderLoader()}
<FadeIn show={!(topStatsLoading || graphLoading)}>
<div id="top-stats-container" className="flex flex-wrap" ref={topStatsBoundary} style={{ height: getTopStatsHeight() }}>
<TopStats site={site} query={query} data={topStatData} onMetricUpdate={onMetricUpdate} tooltipBoundary={topStatsBoundary.current} lastLoadTimestamp={lastLoadTimestamp} />
</div>
<div className="relative px-2">
{mainGraphRefreshing && renderLoader()}
{graphRefreshing && renderLoader()}
<div className="absolute right-4 -top-8 py-1 flex items-center">
{this.downloadLink()}
{this.samplingNotice()}
{this.importedNotice()}
<IntervalPicker site={site} query={query} graphData={graphData} metric={metric} updateInterval={updateInterval} />
{!isRealtime && <StatsExport site={site} query={query} />}
<SamplingNotice samplePercent={topStatData}/>
<WithImportedSwitch site={site} topStatData={topStatData} />
<IntervalPicker site={site} query={query} onIntervalUpdate={onIntervalUpdate} />
</div>
<FadeIn show={graphData}>
<div className="relative h-96 print:h-auto print:pb-8 w-full z-0">
<canvas id="main-graph-canvas" className={canvasClass}></canvas>
</div>
</FadeIn>
<LineGraphWithRouter graphData={graphData} darkTheme={isDarkTheme} query={query} />
</div>
</div>
)
}
}
const LineGraphWithRouter = withRouter(LineGraph)
export default class VisitorGraph extends React.Component {
constructor(props) {
super(props)
this.state = {
topStatsLoadingState: LoadingState.loading,
mainGraphLoadingState: LoadingState.loading,
metric: storage.getItem(`metric__${this.props.site.domain}`) || 'visitors'
}
this.onVisible = this.onVisible.bind(this)
this.updateMetric = this.updateMetric.bind(this)
this.fetchTopStatData = this.fetchTopStatData.bind(this)
this.fetchGraphData = this.fetchGraphData.bind(this)
this.updateInterval = this.updateInterval.bind(this)
}
isIntervalValid(interval) {
const { query, site } = this.props
const validIntervals = site.validIntervalsByPeriod[query.period] || []
return validIntervals.includes(interval)
}
getIntervalFromStorage() {
const { query, site } = this.props
let interval = getStoredInterval(query.period, site.domain)
if (interval !== "week" && interval !== "month" && query.period === "custom" && monthsBetweenDates(query.from, query.to) > 12) {
interval = "month"
}
if (this.isIntervalValid(interval)) {
return interval
} else {
return null
}
}
updateInterval(interval) {
if (this.isIntervalValid(interval)) {
storeInterval(this.props.query.period, this.props.site.domain, interval)
this.setState({ mainGraphLoadingState: LoadingState.refreshing, graphData: null }, this.fetchGraphData)
}
}
onVisible() {
this.setState({ mainGraphLoadingState: LoadingState.loading }, this.fetchGraphData)
this.fetchTopStatData()
if (this.props.query.period === 'realtime') {
document.addEventListener('tick', this.fetchGraphData)
document.addEventListener('tick', this.fetchTopStatData)
}
}
componentDidUpdate(prevProps, prevState) {
const { metric } = this.state;
const { query } = this.props
if (query !== prevProps.query) {
this.setState({ mainGraphLoadingState: LoadingState.loading, topStatsLoadingState: LoadingState.loading, graphData: null, topStatData: null }, this.fetchGraphData)
this.fetchTopStatData()
}
if (metric !== prevState.metric) {
this.setState({ mainGraphLoadingState: LoadingState.refreshing }, this.fetchGraphData)
}
}
resetMetric() {
const { topStatData } = this.state
const { query, site } = this.props
const savedMetric = storage.getItem(`metric__${site.domain}`)
const selectableMetrics = topStatData && topStatData.top_stats.map(({ name }) => METRIC_MAPPING[name]).filter(name => name)
const canSelectSavedMetric = selectableMetrics && selectableMetrics.includes(savedMetric)
if (query.filters.goal && !['conversion_rate', 'events'].includes(savedMetric)) {
this.setState({ metric: 'conversions' })
} else if (canSelectSavedMetric) {
this.setState({ metric: savedMetric })
} else {
this.setState({ metric: 'visitors' })
}
}
componentWillUnmount() {
document.removeEventListener('tick', this.fetchGraphData)
document.removeEventListener('tick', this.fetchTopStatData)
}
storeTopStatsContainerHeight() {
storage.setItem(`topStatsHeight__${this.props.site.domain}`, document.getElementById('top-stats-container').clientHeight)
}
updateMetric(clickedMetric) {
if (this.state.metric == clickedMetric) return
storage.setItem(`metric__${this.props.site.domain}`, clickedMetric)
this.setState({ metric: clickedMetric, graphData: null })
}
fetchGraphData() {
const url = `/api/stats/${encodeURIComponent(this.props.site.domain)}/main-graph`
let params = { metric: this.state.metric }
const interval = this.getIntervalFromStorage()
if (interval) { params.interval = interval }
api.get(url, this.props.query, params)
.then((res) => {
this.setState({ mainGraphLoadingState: LoadingState.loaded, graphData: res })
return res
})
.catch((err) => {
console.log(err)
this.setState({ mainGraphLoadingState: LoadingState.loaded, graphData: false })
})
}
fetchTopStatData() {
const query = { ...this.props.query }
if (!isComparisonEnabled(query.comparison)) query.comparison = 'previous_period'
api.get(`/api/stats/${encodeURIComponent(this.props.site.domain)}/top-stats`, query)
.then((res) => {
this.setState({ topStatsLoadingState: LoadingState.loaded, topStatData: res }, () => {
this.storeTopStatsContainerHeight()
this.resetMetric()
})
return res
})
}
renderInner() {
const { query, site } = this.props;
const { graphData, metric, topStatData, topStatsLoadingState, mainGraphLoadingState } = this.state;
const theme = document.querySelector('html').classList.contains('dark') || false
const mainGraphRefreshing = (mainGraphLoadingState === LoadingState.refreshing)
const topStatAndGraphLoaded = !!(topStatData && graphData)
const shouldShow =
topStatsLoadingState === LoadingState.loaded &&
LoadingState.isLoadedOrRefreshing(mainGraphLoadingState) &&
(topStatData && mainGraphRefreshing || topStatAndGraphLoaded)
return (
<FadeIn show={shouldShow}>
<LineGraphWithRouter mainGraphRefreshing={mainGraphRefreshing} graphData={graphData} topStatData={topStatData} site={site} query={query} darkTheme={theme} metric={metric} updateMetric={this.updateMetric} updateInterval={this.updateInterval} lastLoadTimestamp={this.props.lastLoadTimestamp} />
</FadeIn>
)
}
render() {
const { mainGraphLoadingState, topStatsLoadingState } = this.state
const showLoader =
[mainGraphLoadingState, topStatsLoadingState].includes(LoadingState.loading) &&
mainGraphLoadingState !== LoadingState.refreshing
return (
<LazyLoader onVisible={this.onVisible}>
<div className={"relative w-full mt-2 bg-white rounded shadow-xl dark:bg-gray-825"}>
{showLoader && renderLoader()}
{this.renderInner()}
</div>
</LazyLoader>
)
}
</div>
)
}
function renderLoader() {

@ -0,0 +1,40 @@
import React from "react"
import { parseNaiveDate, isBefore } from '../../util/date'
import { Link } from 'react-router-dom'
import * as url from '../../util/url'
import { BarsArrowUpIcon } from '@heroicons/react/20/solid'
export default function WithImportedSwitch({site, topStatData}) {
if (!topStatData?.imports_exist) {
return null
}
function isBeforeNativeStats(date) {
if (!date) return false
const nativeStatsBegin = parseNaiveDate(site.nativeStatsBegin)
const parsedDate = parseNaiveDate(date)
return isBefore(parsedDate, nativeStatsBegin, "day")
}
const isQueryingImportedPeriod = isBeforeNativeStats(topStatData.from)
const isComparingImportedPeriod = isBeforeNativeStats(topStatData.comparing_from)
if (isQueryingImportedPeriod || isComparingImportedPeriod) {
const withImported = topStatData.with_imported;
const toggleColor = withImported ? " dark:text-gray-300 text-gray-700" : " dark:text-gray-500 text-gray-400"
const target = url.setQuery('with_imported', !withImported)
const tip = withImported ? "" : "do not ";
return (
<Link to={target} className="w-4 h-4 mx-2">
<div tooltip={`Stats ${tip}include imported data.`} className="cursor-pointer w-4 h-4">
<BarsArrowUpIcon className={"absolute " + toggleColor} />
</div>
</Link>
)
} else {
return null
}
}

@ -10,11 +10,11 @@ export const FILTER_GROUPS = {
'utm': ['utm_medium', 'utm_source', 'utm_campaign', 'utm_term', 'utm_content'],
'goal': ['goal'],
'props': ['prop_key', 'prop_value'],
...(flags.hostname_filter ? { 'hostname': ['hostname', 'experimental_hostname_filter'] } : {})
...(flags.hostname_filter ? { 'hostname': ['hostname'] } : {})
}
export const NO_CONTAINS_OPERATOR = new Set(['experimental_hostname_filter', 'goal', 'screen'].concat(FILTER_GROUPS['location']))
export const NO_CONTAINS_OPERATOR = new Set(['goal', 'screen'].concat(FILTER_GROUPS['location']))
export const FILTER_OPERATIONS = {
isNot: 'is not',
@ -29,7 +29,7 @@ export const OPERATION_PREFIX = {
};
export function supportsIsNot(filterName) {
return !['goal', 'prop_key', 'experimental_hostname_filter'].includes(filterName)
return !['goal', 'prop_key'].includes(filterName)
}
export function isFreeChoiceFilter(filterName) {
@ -157,7 +157,6 @@ export const formattedFilters = {
'city': 'City',
'page': 'Page',
'hostname': 'Hostname',
'experimental_hostname_filter': 'Treat hostname as entry/exit hostname',
'entry_page': 'Entry Page',
'exit_page': 'Exit Page',
}

@ -38,7 +38,7 @@ config :tailwind,
config :ua_inspector,
database_path: "priv/ua_inspector",
remote_release: "6.2.1"
remote_release: "6.3.0"
config :ref_inspector,
database_path: "priv/ref_inspector"

@ -72,35 +72,36 @@ defmodule Plausible.DataMigration.SiteImports do
end_date = imported_stats_end_date(site.id, import_ids)
end_date =
if !end_date do
IO.puts(
"Site import #{site_import.id} (site ID #{site.id}) does not have any recorded stats. Setting end date to minimum."
)
if !end_date do
IO.puts(
"Site import #{site_import.id} (site ID #{site.id}) does not have any recorded stats. Removing it."
)
Date.add(site_import.start_date, 2)
else
end_date
end
end_date =
if Date.compare(end_date, site_import.end_date) in [:lt, :eq] do
end_date
else
IO.puts(
"Site import #{site_import.id} (site ID #{site.id}) computed end date is later than the current one. Skipping."
)
site_import.end_date
if site_import.legacy do
clear_imported_data(site, dry_run?)
end
site_import
|> Ecto.Changeset.change(end_date: end_date)
|> update!(dry_run?)
IO.puts(
"End date of site import #{site_import.id} (site ID #{site.id}) adjusted to #{end_date}"
)
delete!(site_import, dry_run?)
else
end_date =
if Date.compare(end_date, site_import.end_date) in [:lt, :eq] do
end_date
else
IO.puts(
"Site import #{site_import.id} (site ID #{site.id}) computed end date is later than the current one. Skipping."
)
site_import.end_date
end
site_import
|> Ecto.Changeset.change(end_date: end_date)
|> update!(dry_run?)
IO.puts(
"End date of site import #{site_import.id} (site ID #{site.id}) adjusted to #{end_date}"
)
end
end
IO.puts("Done processing site ID #{site.id}")
@ -149,6 +150,14 @@ defmodule Plausible.DataMigration.SiteImports do
end
end
defp clear_imported_data(site, false = _dry_run?) do
Repo.update_all(from(s in Site, where: s.id == ^site.id), set: [imported_data: nil])
end
defp clear_imported_data(site, true = _dry_run?) do
%{site | imported_data: nil}
end
defp update!(changeset, false = _dry_run?) do
Repo.update!(changeset)
end
@ -161,6 +170,14 @@ defmodule Plausible.DataMigration.SiteImports do
end
end
defp delete!(entity, false = _dry_run?) do
Repo.delete!(entity)
end
defp delete!(entity, true = _dry_run?) do
entity
end
defp max_date_query(schema, site_id, import_ids) do
from(q in schema,
where: q.site_id == ^site_id,

@ -80,6 +80,14 @@ defmodule Plausible.Goals do
def find_or_create(_, %{"goal_type" => "page"}), do: {:missing, "page_path"}
def list_revenue_goals(site) do
from(g in Plausible.Goal,
where: g.site_id == ^site.id and not is_nil(g.currency),
select: %{event_name: g.event_name, currency: g.currency}
)
|> Plausible.Repo.all()
end
def for_site(site, opts \\ []) do
site
|> for_site_query(opts)

@ -140,7 +140,10 @@ defmodule Plausible.Google.GA4.HTTP do
{:error, %HTTPClient.Non200Error{} = error} when error.reason.status in [401, 403] ->
{:error, :authentication_failed}
{:error, %HTTPClient.Non200Error{} = error} ->
{:error, %{reason: :timeout}} ->
{:error, :timeout}
{:error, error} ->
Sentry.capture_message("Error listing GA4 accounts for user", extra: %{error: error})
{:error, :unknown}
end
@ -161,7 +164,10 @@ defmodule Plausible.Google.GA4.HTTP do
{:error, %HTTPClient.Non200Error{} = error} when error.reason.status in [404] ->
{:error, :not_found}
{:error, %HTTPClient.Non200Error{} = error} ->
{:error, %{reason: :timeout}} ->
{:error, :timeout}
{:error, error} ->
Sentry.capture_message("Error retrieving GA4 property #{property}",
extra: %{error: error}
)
@ -221,7 +227,10 @@ defmodule Plausible.Google.GA4.HTTP do
{:error, %HTTPClient.Non200Error{} = error} when error.reason.status in [401, 403] ->
{:error, :authentication_failed}
{:error, %HTTPClient.Non200Error{} = error} ->
{:error, %{reason: :timeout}} ->
{:error, :timeout}
{:error, error} ->
Sentry.capture_message("Error retrieving GA4 #{edge} date",
extra: %{error: error}
)

@ -108,7 +108,10 @@ defmodule Plausible.Google.UA.HTTP do
{:error, %HTTPClient.Non200Error{} = error} when error.reason.status in [401, 403] ->
{:error, :authentication_failed}
{:error, %HTTPClient.Non200Error{} = error} ->
{:error, %{reason: :timeout}} ->
{:error, :timeout}
{:error, error} ->
Sentry.capture_message("Error listing UA views for user", extra: %{error: error})
{:error, :unknown}
end
@ -170,7 +173,10 @@ defmodule Plausible.Google.UA.HTTP do
{:error, %HTTPClient.Non200Error{} = error} when error.reason.status in [401, 403] ->
{:error, :authentication_failed}
{:error, %HTTPClient.Non200Error{} = error} ->
{:error, %{reason: :timeout}} ->
{:error, :timeout}
{:error, error} ->
Sentry.capture_message("Error retrieving UA #{edge} date",
extra: %{error: error}
)

@ -62,9 +62,9 @@ defmodule Plausible.Imported do
}
end
@spec get_import(non_neg_integer()) :: SiteImport.t() | nil
def get_import(import_id) do
Repo.get(SiteImport, import_id)
@spec get_import(Site.t(), non_neg_integer()) :: SiteImport.t() | nil
def get_import(site, import_id) do
Repo.get_by(SiteImport, id: import_id, site_id: site.id)
end
defdelegate listen(), to: Imported.Importer
@ -83,6 +83,16 @@ defmodule Plausible.Imported do
end
end
@spec other_imports_in_progress?(SiteImport.t()) :: boolean()
def other_imports_in_progress?(site_import) do
Repo.exists?(
from(i in SiteImport,
where: i.site_id == ^site_import.site_id and i.id != ^site_import.id,
where: i.status in ^[SiteImport.pending(), SiteImport.importing()]
)
)
end
defp maybe_filter_by_status(query, nil), do: query
defp maybe_filter_by_status(query, status) do

@ -91,6 +91,7 @@ defmodule Plausible.Imported.Importer do
like LiveView, where notifications can be listened for via `handle_info/2`.
"""
alias Plausible.Imported
alias Plausible.Imported.SiteImport
alias Plausible.Repo
@ -98,7 +99,7 @@ defmodule Plausible.Imported.Importer do
@callback label() :: String.t()
@callback email_template() :: String.t()
@callback parse_args(map()) :: Keyword.t()
@callback import_data(Plausible.Imported.SiteImport.t(), Keyword.t()) :: :ok | {:error, any()}
@callback import_data(SiteImport.t(), Keyword.t()) :: :ok | {:error, any()}
@callback before_start(SiteImport.t()) :: :ok | {:ok, map()} | {:error, any()}
@callback on_success(SiteImport.t(), map()) :: :ok
@callback on_failure(SiteImport.t()) :: :ok
@ -108,7 +109,7 @@ defmodule Plausible.Imported.Importer do
@behaviour Plausible.Imported.Importer
@spec new_import(Plausible.Site.t(), Plausible.Auth.User.t(), Keyword.t()) ::
{:ok, Oban.Job.t()} | {:error, Ecto.Changeset.t()}
{:ok, Oban.Job.t()} | {:error, Ecto.Changeset.t() | :import_in_progress}
def new_import(site, user, opts) do
Plausible.Imported.Importer.new_import(name(), site, user, opts, &before_start/1)
end
@ -222,11 +223,15 @@ defmodule Plausible.Imported.Importer do
:ok = listen()
end
opts
|> Keyword.put(:import_id, site_import.id)
|> Map.new()
|> Plausible.Workers.ImportAnalytics.new()
|> Oban.insert()
if not Imported.other_imports_in_progress?(site_import) do
opts
|> Keyword.put(:import_id, site_import.id)
|> Map.new()
|> Plausible.Workers.ImportAnalytics.new()
|> Oban.insert()
else
{:error, :import_in_progress}
end
end
defp mark_complete(site_import, extra_data, on_success_fun) do

@ -5,19 +5,6 @@ defmodule Plausible.Stats.Aggregate do
import Ecto.Query
alias Plausible.Stats.{Query, Util}
@revenue_metrics on_full_build(do: Plausible.Stats.Goal.Revenue.revenue_metrics(), else: [])
@event_metrics [
:visitors,
:pageviews,
:events,
:sample_percent,
:conversion_rate,
:total_visitors
] ++ @revenue_metrics
@session_metrics [:visits, :bounce_rate, :visit_duration, :views_per_visit, :sample_percent]
def aggregate(site, query, metrics) do
{currency, metrics} =
on_full_build do
@ -28,18 +15,17 @@ defmodule Plausible.Stats.Aggregate do
Query.trace(query, metrics)
event_metrics =
{event_metrics, session_metrics, other_metrics} =
metrics
|> Util.maybe_add_visitors_metric()
|> Enum.filter(&(&1 in @event_metrics))
|> Plausible.Stats.TableDecider.partition_metrics(query)
event_task = fn -> aggregate_events(site, query, event_metrics) end
session_metrics = Enum.filter(metrics, &(&1 in @session_metrics))
session_task = fn -> aggregate_sessions(site, query, session_metrics) end
time_on_page_task =
if :time_on_page in metrics do
if :time_on_page in other_metrics do
fn -> aggregate_time_on_page(site, query) end
else
fn -> %{} end

@ -3,7 +3,7 @@ defmodule Plausible.Stats.Base do
use Plausible
use Plausible.Stats.Fragments
alias Plausible.Stats.{Query, Filters}
alias Plausible.Stats.{Query, Filters, TableDecider}
alias Plausible.Timezones
import Ecto.Query
@ -15,7 +15,7 @@ defmodule Plausible.Stats.Base do
def base_event_query(site, query) do
events_q = query_events(site, query)
if Enum.any?(Filters.visit_props(), &query.filters["visit:" <> &1]) do
if TableDecider.events_join_sessions?(query) do
sessions_q =
from(
s in query_sessions(site, query),
@ -30,11 +30,15 @@ defmodule Plausible.Stats.Base do
on: e.session_id == sq.session_id
)
else
events_q
if query.experimental_reduced_joins? do
events_q |> filter_by_visit_props(Filters.event_table_visit_props(), query)
else
events_q
end
end
end
def query_events(site, query) do
defp query_events(site, query) do
{first_datetime, last_datetime} = utc_boundaries(query, site)
q =
@ -125,18 +129,6 @@ defmodule Plausible.Stats.Base do
q
end
@api_prop_name_to_db %{
"source" => "referrer_source",
"device" => "screen_size",
"screen" => "screen_size",
"os" => "operating_system",
"os_version" => "operating_system_version",
"country" => "country_code",
"region" => "subdivision1_code",
"city" => "city_geoname_id",
"entry_page_hostname" => "hostname"
}
def query_sessions(site, query) do
{first_datetime, last_datetime} =
utc_boundaries(query, site)
@ -144,8 +136,7 @@ defmodule Plausible.Stats.Base do
q = from(s in "sessions_v2", where: s.site_id == ^site.id)
sessions_q =
if FunWithFlags.enabled?(:experimental_session_count, for: site) or
query.experimental_session_count? do
if query.experimental_session_count? do
from s in q, where: s.timestamp >= ^first_datetime and s.start < ^last_datetime
else
from s in q, where: s.start >= ^first_datetime and s.start < ^last_datetime
@ -155,9 +146,24 @@ defmodule Plausible.Stats.Base do
sessions_q = Plausible.Stats.Sampling.add_query_hint(sessions_q, query)
end
sessions_q = filter_by_entry_props(sessions_q, query)
filter_by_entry_props(sessions_q, query)
|> filter_by_visit_props(Filters.visit_props(), query)
end
Enum.reduce(Filters.visit_props(), sessions_q, fn prop_name, sessions_q ->
@api_prop_name_to_db %{
"source" => "referrer_source",
"device" => "screen_size",
"screen" => "screen_size",
"os" => "operating_system",
"os_version" => "operating_system_version",
"country" => "country_code",
"region" => "subdivision1_code",
"city" => "city_geoname_id",
"entry_page_hostname" => "hostname"
}
defp filter_by_visit_props(q, visit_props, query) do
Enum.reduce(visit_props, q, fn prop_name, sessions_q ->
filter_key = "visit:" <> prop_name
db_field =
@ -241,6 +247,13 @@ defmodule Plausible.Stats.Base do
}
end
defp select_event_metric(:visits) do
%{
visits:
dynamic([e], fragment("toUInt64(round(uniq(?) * any(_sample_factor)))", e.session_id))
}
end
on_full_build do
defp select_event_metric(:total_revenue) do
%{total_revenue: Plausible.Stats.Goal.Revenue.total_revenue_query()}

@ -6,7 +6,7 @@ defmodule Plausible.Stats.Breakdown do
import Plausible.Stats.{Base, Imported}
import Ecto.Query
require OpenTelemetry.Tracer, as: Tracer
alias Plausible.Stats.{Query, Util}
alias Plausible.Stats.{Query, Util, TableDecider}
@no_ref "Direct / None"
@not_set "(not set)"
@ -103,7 +103,6 @@ defmodule Plausible.Stats.Breakdown do
{event_q, page_q} ->
from(
e in subquery(union_all(event_q, ^page_q)),
# :TODO: Handle other orderings
order_by: [desc: e.visitors]
)
|> apply_pagination(pagination)
@ -200,9 +199,35 @@ defmodule Plausible.Stats.Breakdown do
metrics_to_select = Util.maybe_add_visitors_metric(metrics) -- @computed_metrics
breakdown_sessions(site, query, property, metrics_to_select)
|> maybe_add_group_conversion_rate(&breakdown_sessions/4, site, query, property, metrics)
|> paginate_and_execute(metrics, pagination)
case breakdown_table(query, metrics, property) do
:session ->
breakdown_sessions(site, query, property, metrics_to_select)
|> maybe_add_group_conversion_rate(&breakdown_sessions/4, site, query, property, metrics)
|> paginate_and_execute(metrics, pagination)
:event ->
breakdown_events(site, query, property, metrics_to_select)
|> maybe_add_group_conversion_rate(&breakdown_events/4, site, query, property, metrics)
|> paginate_and_execute(metrics, pagination)
end
end
# Backwards compatibility
# defp breakdown_table(%Query{experimental_reduced_joins?: false}, _, _), do: :session
defp breakdown_table(_query, _metrics, "visit:entry_page"), do: :session
defp breakdown_table(_query, _metrics, "visit:entry_page_hostname"), do: :session
defp breakdown_table(_query, _metrics, "visit:exit_page"), do: :session
defp breakdown_table(_query, _metrics, "visit:exit_page_hostname"), do: :session
defp breakdown_table(query, metrics, property) do
{_, session_metrics, _} = TableDecider.partition_metrics(metrics, query, property)
if not Enum.empty?(session_metrics) do
:session
else
:event
end
end
defp zip_results(event_result, session_result, property, metrics) do

@ -118,10 +118,6 @@ defmodule Plausible.Stats.FilterSuggestions do
end)
end
def filter_suggestions(_site, _query, "experimental_hostname_filter", _filter_search) do
wrap_suggestions(["true", "false"])
end
def filter_suggestions(site, _query, "goal", filter_search) do
site
|> Plausible.Goals.for_site()

@ -29,6 +29,15 @@ defmodule Plausible.Stats.Filters do
]
def visit_props(), do: @visit_props |> Enum.map(&to_string/1)
@event_table_visit_props @visit_props --
[
:entry_page,
:exit_page,
:entry_page_hostname,
:exit_page_hostname
]
def event_table_visit_props(), do: @event_table_visit_props |> Enum.map(&to_string/1)
@event_props [:name, :page, :goal, :hostname]
def event_props(), do: @event_props |> Enum.map(&to_string/1)

@ -10,7 +10,7 @@ defmodule Plausible.Stats.Query do
include_imported: false,
now: nil,
experimental_session_count?: false,
experimental_hostname_filter?: false
experimental_reduced_joins?: false
require OpenTelemetry.Tracer, as: Tracer
alias Plausible.Stats.{Filters, Interval}
@ -23,7 +23,8 @@ defmodule Plausible.Stats.Query do
query =
__MODULE__
|> struct!(now: now)
|> put_experimental_flags(params)
|> put_experimental_session_count(site, params)
|> put_experimental_reduced_joins(site, params)
|> put_period(site, params)
|> put_interval(params)
|> put_parsed_filters(params)
@ -37,14 +38,28 @@ defmodule Plausible.Stats.Query do
query
end
defp put_experimental_flags(query, params) do
%{
"experimental_session_count" => :experimental_session_count?,
"experimental_hostname_filter" => :experimental_hostname_filter?
}
|> Enum.reduce(query, fn {param, flag}, query ->
if Map.get(params, param) == "true", do: Map.put(query, flag, true), else: query
end)
defp put_experimental_session_count(query, site, params) do
if Map.has_key?(params, "experimental_session_count") do
struct!(query,
experimental_session_count?: Map.get(params, "experimental_session_count") == "true"
)
else
struct!(query,
experimental_session_count?: FunWithFlags.enabled?(:experimental_session_count, for: site)
)
end
end
defp put_experimental_reduced_joins(query, site, params) do
if Map.has_key?(params, "experimental_reduced_joins") do
struct!(query,
experimental_reduced_joins?: Map.get(params, "experimental_reduced_joins") == "true"
)
else
struct!(query,
experimental_reduced_joins?: FunWithFlags.enabled?(:experimental_reduced_joins, for: site)
)
end
end
defp put_period(query, site, %{"period" => "realtime"}) do

@ -0,0 +1,111 @@
defmodule Plausible.Stats.TableDecider do
@moduledoc """
This module contains logic for deciding which tables need to be queried given a query
and metrics, with the purpose of reducing the number of queries and JOINs needed to perform.
"""
import Enum, only: [empty?: 1]
alias Plausible.Stats.Query
def events_join_sessions?(query) do
Enum.any?(query.filters, &(filters_partitioner(query, &1) == :session))
end
def partition_metrics(metrics, query, breakdown_property \\ nil) do
%{
event: event_only_metrics,
session: session_only_metrics,
either: either_metrics,
other: other_metrics,
sample_percent: sample_percent
} =
partition(metrics, query, &metric_partitioner/2)
# Treat breakdown property as yet another filter
filters =
if breakdown_property do
Map.put(query.filters, breakdown_property, nil)
else
query.filters
end
%{event: event_only_filters, session: session_only_filters} =
partition(filters, query, &filters_partitioner/2)
cond do
# Only one table needs to be queried
empty?(event_only_metrics) && empty?(event_only_filters) ->
{[], session_only_metrics ++ either_metrics ++ sample_percent, other_metrics}
empty?(session_only_metrics) && empty?(session_only_filters) ->
{event_only_metrics ++ either_metrics ++ sample_percent, [], other_metrics}
# Filters on both events and sessions, but only one kind of metric
empty?(event_only_metrics) ->
{[], session_only_metrics ++ either_metrics ++ sample_percent, other_metrics}
empty?(session_only_metrics) ->
{event_only_metrics ++ either_metrics ++ sample_percent, [], other_metrics}
# Default: prefer sessions
true ->
{event_only_metrics ++ sample_percent,
session_only_metrics ++ either_metrics ++ sample_percent, other_metrics}
end
end
defp metric_partitioner(_, :conversion_rate), do: :event
defp metric_partitioner(_, :average_revenue), do: :event
defp metric_partitioner(_, :total_revenue), do: :event
defp metric_partitioner(_, :pageviews), do: :event
defp metric_partitioner(_, :events), do: :event
defp metric_partitioner(_, :bounce_rate), do: :session
defp metric_partitioner(_, :visit_duration), do: :session
defp metric_partitioner(_, :views_per_visit), do: :session
# Metrics which used to only be queried from one table but can be calculated from either
defp metric_partitioner(%Query{experimental_reduced_joins?: true}, :visits), do: :either
defp metric_partitioner(%Query{experimental_reduced_joins?: true}, :visitors), do: :either
defp metric_partitioner(_, :visits), do: :session
defp metric_partitioner(_, :visitors), do: :event
# Calculated metrics - handled on callsite separately from other metrics.
defp metric_partitioner(_, :time_on_page), do: :other
defp metric_partitioner(_, :total_visitors), do: :other
defp metric_partitioner(_, :percentage), do: :other
# Sample percentage is included in both tables if queried.
defp metric_partitioner(_, :sample_percent), do: :sample_percent
defp metric_partitioner(%Query{experimental_reduced_joins?: false}, unknown) do
raise ArgumentError, "Metric #{unknown} not supported without experimental_reduced_joins?"
end
defp metric_partitioner(_, _), do: :either
defp filters_partitioner(_, {"event:" <> _, _}), do: :event
defp filters_partitioner(_, {"visit:entry_page", _}), do: :session
defp filters_partitioner(_, {"visit:entry_page_hostname", _}), do: :session
defp filters_partitioner(_, {"visit:exit_page", _}), do: :session
defp filters_partitioner(_, {"visit:exit_page_hostname", _}), do: :session
defp filters_partitioner(%Query{experimental_reduced_joins?: true}, {"visit:" <> _, _}),
do: :either
defp filters_partitioner(_, {"visit:" <> _, _}),
do: :session
defp filters_partitioner(%Query{experimental_reduced_joins?: false}, {unknown, _}) do
raise ArgumentError, "Filter #{unknown} not supported without experimental_reduced_joins?"
end
defp filters_partitioner(_, _), do: :either
@default %{event: [], session: [], either: [], other: [], sample_percent: []}
defp partition(values, query, partitioner) do
Enum.reduce(values, @default, fn value, acc ->
key = partitioner.(query, value)
Map.put(acc, key, Map.fetch!(acc, key) ++ [value])
end)
end
end

@ -18,15 +18,11 @@ defmodule Plausible.Stats.Timeseries do
@typep value :: nil | integer() | float()
@type results :: nonempty_list(%{required(:date) => Date.t(), required(metric()) => value()})
@revenue_metrics on_full_build(do: Plausible.Stats.Goal.Revenue.revenue_metrics(), else: [])
@event_metrics [:visitors, :pageviews, :events, :conversion_rate] ++ @revenue_metrics
@session_metrics [:visits, :bounce_rate, :visit_duration, :views_per_visit]
def timeseries(site, query, metrics) do
steps = buckets(query)
event_metrics = Enum.filter(metrics, &(&1 in @event_metrics))
session_metrics = Enum.filter(metrics, &(&1 in @session_metrics))
{event_metrics, session_metrics, _} =
Plausible.Stats.TableDecider.partition_metrics(metrics, query)
{currency, event_metrics} =
on_full_build do

@ -129,6 +129,7 @@ defmodule PlausibleWeb.Api.StatsController do
full_intervals = build_full_intervals(query, labels)
json(conn, %{
metric: metric,
plot: plot_timeseries(timeseries_result, metric),
labels: labels,
comparison_plot: comparison_result && plot_timeseries(comparison_result, metric),
@ -294,10 +295,12 @@ defmodule PlausibleWeb.Api.StatsController do
},
%{
name: "Unique conversions (last 30 min)",
graph_metric: :visitors,
value: unique_conversions
},
%{
name: "Total conversions (last 30 min)",
graph_metric: :events,
value: total_conversions
}
]
@ -320,10 +323,12 @@ defmodule PlausibleWeb.Api.StatsController do
},
%{
name: "Unique visitors (last 30 min)",
graph_metric: :visitors,
value: visitors
},
%{
name: "Pageviews (last 30 min)",
graph_metric: :pageviews,
value: pageviews
}
]
@ -340,15 +345,21 @@ defmodule PlausibleWeb.Api.StatsController do
[
top_stats_entry(results, comparison, "Unique visitors", :total_visitors),
top_stats_entry(results, comparison, "Unique conversions", :visitors),
top_stats_entry(results, comparison, "Total conversions", :events),
top_stats_entry(results, comparison, "Unique conversions", :visitors, graphable?: true),
top_stats_entry(results, comparison, "Total conversions", :events, graphable?: true),
on_full_build do
top_stats_entry(results, comparison, "Average revenue", :average_revenue, &format_money/1)
top_stats_entry(results, comparison, "Average revenue", :average_revenue,
formatter: &format_money/1,
graphable?: true
)
end,
on_full_build do
top_stats_entry(results, comparison, "Total revenue", :total_revenue, &format_money/1)
top_stats_entry(results, comparison, "Total revenue", :total_revenue,
formatter: &format_money/1,
graphable?: true
)
end,
top_stats_entry(results, comparison, "Conversion rate", :conversion_rate)
top_stats_entry(results, comparison, "Conversion rate", :conversion_rate, graphable?: true)
]
|> Enum.reject(&is_nil/1)
|> then(&{&1, 100})
@ -382,39 +393,63 @@ defmodule PlausibleWeb.Api.StatsController do
stats =
[
top_stats_entry(current_results, prev_results, "Unique visitors", :visitors),
top_stats_entry(current_results, prev_results, "Total visits", :visits),
top_stats_entry(current_results, prev_results, "Total pageviews", :pageviews),
top_stats_entry(current_results, prev_results, "Views per visit", :views_per_visit),
top_stats_entry(current_results, prev_results, "Bounce rate", :bounce_rate),
top_stats_entry(current_results, prev_results, "Visit duration", :visit_duration),
top_stats_entry(current_results, prev_results, "Time on page", :time_on_page, fn
nil -> 0
value -> value
end)
top_stats_entry(current_results, prev_results, "Unique visitors", :visitors,
graphable?: true
),
top_stats_entry(current_results, prev_results, "Total visits", :visits, graphable?: true),
top_stats_entry(current_results, prev_results, "Total pageviews", :pageviews,
graphable?: true
),
top_stats_entry(current_results, prev_results, "Views per visit", :views_per_visit,
graphable?: true
),
top_stats_entry(current_results, prev_results, "Bounce rate", :bounce_rate,
graphable?: true
),
top_stats_entry(current_results, prev_results, "Visit duration", :visit_duration,
graphable?: true
),
top_stats_entry(current_results, prev_results, "Time on page", :time_on_page,
formatter: fn
nil -> 0
value -> value
end
)
]
|> Enum.filter(& &1)
{stats, current_results[:sample_percent][:value]}
end
defp top_stats_entry(current_results, prev_results, name, key, formatter \\ & &1) do
defp top_stats_entry(current_results, prev_results, name, key, opts \\ []) do
if current_results[key] do
formatter = Keyword.get(opts, :formatter, & &1)
value = get_in(current_results, [key, :value])
if prev_results do
prev_value = get_in(prev_results, [key, :value])
change = Stats.Compare.calculate_change(key, prev_value, value)
%{name: name, value: formatter.(value)}
|> maybe_put_graph_metric(opts, key)
|> maybe_put_comparison(prev_results, key, value, formatter)
end
end
%{
name: name,
value: formatter.(value),
comparison_value: formatter.(prev_value),
change: change
}
else
%{name: name, value: formatter.(value)}
end
defp maybe_put_graph_metric(entry, opts, key) do
if Keyword.get(opts, :graphable?) do
entry |> Map.put(:graph_metric, key)
else
entry
end
end
defp maybe_put_comparison(entry, prev_results, key, value, formatter) do
if prev_results do
prev_value = get_in(prev_results, [key, :value])
change = Stats.Compare.calculate_change(key, prev_value, value)
entry
|> Map.put(:comparison_value, formatter.(prev_value))
|> Map.put(:change, change)
else
entry
end
end
@ -424,7 +459,7 @@ defmodule PlausibleWeb.Api.StatsController do
pagination = parse_pagination(params)
query =
if query.experimental_hostname_filter? and query.filters["event:hostname"] do
if query.filters["event:hostname"] do
Query.put_filter(query, "visit:entry_page_hostname", query.filters["event:hostname"])
else
query
@ -736,7 +771,7 @@ defmodule PlausibleWeb.Api.StatsController do
metrics = breakdown_metrics(query, [:visits, :visit_duration])
query =
if query.experimental_hostname_filter? and query.filters["event:hostname"] do
if query.filters["event:hostname"] do
Query.put_filter(query, "visit:entry_page_hostname", query.filters["event:hostname"])
else
query
@ -773,7 +808,7 @@ defmodule PlausibleWeb.Api.StatsController do
metrics = breakdown_metrics(query, [:visits])
query =
if query.experimental_hostname_filter? and query.filters["event:hostname"] do
if query.filters["event:hostname"] do
Query.put_filter(query, "visit:exit_page_hostname", query.filters["event:hostname"])
else
query

@ -96,6 +96,14 @@ defmodule PlausibleWeb.GoogleAnalyticsController do
)
|> redirect(external: redirect_route)
{:error, :timeout} ->
conn
|> put_flash(
:error,
"Google Analytics API has timed out. Please try again."
)
|> redirect(external: redirect_route)
{:error, _any} ->
conn
|> put_flash(
@ -168,6 +176,14 @@ defmodule PlausibleWeb.GoogleAnalyticsController do
)
|> redirect(external: redirect_route)
{:error, :timeout} ->
conn
|> put_flash(
:error,
"Google Analytics API has timed out. Please try again."
)
|> redirect(external: redirect_route)
{:error, _any} ->
conn
|> put_flash(
@ -225,6 +241,14 @@ defmodule PlausibleWeb.GoogleAnalyticsController do
)
|> redirect(external: redirect_route)
{:error, :timeout} ->
conn
|> put_flash(
:error,
"Google Analytics API has timed out. Please try again."
)
|> redirect(external: redirect_route)
{:error, :not_found} ->
conn
|> put_flash(
@ -265,48 +289,48 @@ defmodule PlausibleWeb.GoogleAnalyticsController do
Routes.site_path(conn, :settings_imports_exports, site.domain)
end
case Imported.check_dates(site, start_date, end_date) do
{:ok, start_date, end_date} ->
if Google.API.property?(property_or_view) do
{:ok, _} =
Imported.GoogleAnalytics4.new_import(
site,
current_user,
property: property_or_view,
label: property_or_view,
start_date: start_date,
end_date: end_date,
access_token: access_token,
refresh_token: refresh_token,
token_expires_at: expires_at
)
else
{:ok, _} =
Imported.UniversalAnalytics.new_import(
site,
current_user,
view_id: property_or_view,
label: property_or_view,
start_date: start_date,
end_date: end_date,
access_token: access_token,
refresh_token: refresh_token,
token_expires_at: expires_at,
legacy: legacy == "true"
)
end
import_opts = [
label: property_or_view,
start_date: start_date,
end_date: end_date,
access_token: access_token,
refresh_token: refresh_token,
token_expires_at: expires_at,
legacy: legacy == "true"
]
with {:ok, start_date, end_date} <- Imported.check_dates(site, start_date, end_date),
import_opts = [{:start_date, start_date}, {:end_date, end_date} | import_opts],
{:ok, _} <- schedule_job(site, current_user, property_or_view, import_opts) do
conn
|> put_flash(:success, "Import scheduled. An email will be sent when it completes.")
|> redirect(external: redirect_route)
else
{:error, :no_time_window} ->
conn
|> put_flash(:success, "Import scheduled. An email will be sent when it completes.")
|> put_flash(
:error,
"Import failed. No data could be imported because date range overlaps with existing data."
)
|> redirect(external: redirect_route)
{:error, :no_time_window} ->
{:error, :import_in_progress} ->
conn
|> put_flash(
:error,
"Import failed. No data could be imported because date range overlaps with existing data."
"There's another import still in progress. Please wait until it's completed or cancel it before starting a new one."
)
|> redirect(external: redirect_route)
end
end
defp schedule_job(site, current_user, property_or_view, opts) do
if Google.API.property?(property_or_view) do
opts = Keyword.put(opts, :property, property_or_view)
Imported.GoogleAnalytics4.new_import(site, current_user, opts)
else
opts = Keyword.put(opts, :view_id, property_or_view)
Imported.UniversalAnalytics.new_import(site, current_user, opts)
end
end
end

@ -652,7 +652,7 @@ defmodule PlausibleWeb.SiteController do
|> Plausible.Site.remove_imported_data()
|> Repo.update!()
site_import = Plausible.Imported.get_import(import_id) ->
site_import = Plausible.Imported.get_import(site, import_id) ->
Oban.cancel_all_jobs(
from(j in Oban.Job,
where:

@ -64,6 +64,7 @@ defmodule PlausibleWeb.StatsController do
|> render("stats.html",
site: site,
has_goals: Plausible.Sites.has_goals?(site),
revenue_goals: list_revenue_goals(site),
funnels: list_funnels(site),
has_props: Plausible.Props.configured?(site),
stats_start_date: stats_start_date,
@ -92,10 +93,13 @@ defmodule PlausibleWeb.StatsController do
defp list_funnels(site) do
Plausible.Funnels.list(site)
end
else
defp list_funnels(_site) do
[]
defp list_revenue_goals(site) do
Plausible.Goals.list_revenue_goals(site)
end
else
defp list_funnels(_site), do: []
defp list_revenue_goals(_site), do: []
end
@doc """
@ -319,6 +323,7 @@ defmodule PlausibleWeb.StatsController do
|> render("stats.html",
site: shared_link.site,
has_goals: Sites.has_goals?(shared_link.site),
revenue_goals: list_revenue_goals(shared_link.site),
funnels: list_funnels(shared_link.site),
has_props: Plausible.Props.configured?(shared_link.site),
stats_start_date: stats_start_date,

@ -42,7 +42,39 @@ defmodule PlausibleWeb.Live.ImportsExportsSettings do
end
def render(assigns) do
import_in_progress? =
Enum.any?(
assigns.site_imports,
&(&1.live_status in [SiteImport.pending(), SiteImport.importing()])
)
assigns = assign(assigns, :import_in_progress?, import_in_progress?)
~H"""
<div class="mt-5 flex gap-x-4">
<.button_link
class="w-36 h-20"
theme="bright"
disabled={@import_in_progress?}
href={Plausible.Google.API.import_authorize_url(@site.id, "import", legacy: false)}
>
<img src="/images/icon/google_analytics_logo.svg" alt="Google Analytics import" />
</.button_link>
<.button_link
class="w-36 h-20"
theme="bright"
disabled={@import_in_progress?}
href={"/#{URI.encode_www_form(@site.domain)}/settings/import"}
>
<img class="h-16" src="/images/icon/csv_logo.svg" alt="New CSV import" />
</.button_link>
</div>
<p :if={@import_in_progress?} class="mt-4 text-red-400 text-sm">
No new imports can be started until the import in progress is completed or cancelled.
</p>
<header class="relative border-b border-gray-200 pb-4">
<h3 class="mt-8 text-md leading-6 font-medium text-gray-900 dark:text-gray-100">
Existing Imports

@ -10,24 +10,6 @@
<PlausibleWeb.Components.Generic.docs_info slug="google-analytics-import" />
</header>
<div class="mt-5 flex gap-x-4">
<PlausibleWeb.Components.Generic.button_link
class="w-36 h-20"
theme="bright"
href={Plausible.Google.API.import_authorize_url(@site.id, "import", legacy: false)}
>
<img src="/images/icon/google_analytics_logo.svg" alt="Google Analytics import" />
</PlausibleWeb.Components.Generic.button_link>
<PlausibleWeb.Components.Generic.button_link
class="w-36 h-20"
theme="bright"
href={"/#{URI.encode_www_form(@site.domain)}/settings/import"}
>
<img class="h-16" src="/images/icon/csv_logo.svg" alt="New CSV import" />
</PlausibleWeb.Components.Generic.button_link>
</div>
<%= live_render(@conn, PlausibleWeb.Live.ImportsExportsSettings,
session: %{"domain" => @site.domain}
) %>

@ -28,11 +28,11 @@
Please subscribe to the appropriate tier with the link below to access the stats again.
</p>
<p class="mt-6 text-sm text-gray-500">
You can check the
You can configure your
<.styled_link href={Routes.site_path(@conn, :settings_general, @site.domain)}>
site settings
</.styled_link>
and we're still counting the stats but your access to the dashboard is restricted.
but your access to the dashboard is restricted.
</p>
</div>
<div class="mt-6 w-full text-center">

@ -27,6 +27,7 @@
data-props-available={
to_string(Plausible.Billing.Feature.Props.check_availability(@site.owner) == :ok)
}
data-revenue-goals={Jason.encode!(@revenue_goals)}
data-funnels={Jason.encode!(@funnels)}
data-has-props={to_string(@has_props)}
data-logged-in={to_string(!!@conn.assigns[:current_user])}

@ -147,7 +147,7 @@
"timex": {:hex, :timex, "3.7.11", "bb95cb4eb1d06e27346325de506bcc6c30f9c6dea40d1ebe390b262fad1862d1", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.20", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 1.1", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm", "8b9024f7efbabaf9bd7aa04f65cf8dcd7c9818ca5737677c7b76acbc6a94d1aa"},
"tls_certificate_check": {:hex, :tls_certificate_check, "1.21.0", "042ab2c0c860652bc5cf69c94e3a31f96676d14682e22ec7813bd173ceff1788", [:rebar3], [{:ssl_verify_fun, "~> 1.1", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm", "6cee6cffc35a390840d48d463541d50746a7b0e421acaadb833cfc7961e490e7"},
"tzdata": {:hex, :tzdata, "1.1.1", "20c8043476dfda8504952d00adac41c6eda23912278add38edc140ae0c5bcc46", [:mix], [{:hackney, "~> 1.17", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm", "a69cec8352eafcd2e198dea28a34113b60fdc6cb57eb5ad65c10292a6ba89787"},
"ua_inspector": {:hex, :ua_inspector, "3.8.0", "c0b0d13200a9bd509225f15ea8cf275c0bec27390a21c355746ff8b8a88c3e4d", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}, {:yamerl, "~> 0.7", [hex: :yamerl, repo: "hexpm", optional: false]}], "hexpm", "7c980bae82a4754075b933e0f383935a681e5a2628856ad3ecf6eb80d8139539"},
"ua_inspector": {:hex, :ua_inspector, "3.9.0", "2021bbddb1ee41f202da7f006fb09f5c5617ad579108b7b9bcf1881828462f04", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}, {:yamerl, "~> 0.7", [hex: :yamerl, repo: "hexpm", optional: false]}], "hexpm", "0f2d1b9806e78b14b91b6f20eaaa4a68f8989f4f2a99a376b874c295ac50a30d"},
"unicode_util_compat": {:hex, :unicode_util_compat, "0.7.0", "bc84380c9ab48177092f43ac89e4dfa2c6d62b40b8bd132b1059ecc7232f9a78", [:rebar3], [], "hexpm", "25eee6d67df61960cf6a794239566599b09e17e668d3700247bc498638152521"},
"uuid": {:hex, :uuid, "1.1.8", "e22fc04499de0de3ed1116b770c7737779f226ceefa0badb3592e64d5cfb4eb9", [:mix], [], "hexpm", "c790593b4c3b601f5dc2378baae7efaf5b3d73c4c6456ba85759905be792f2ac"},
"websock": {:hex, :websock, "0.5.3", "2f69a6ebe810328555b6fe5c831a851f485e303a7c8ce6c5f675abeb20ebdadc", [:mix], [], "hexpm", "6105453d7fac22c712ad66fab1d45abdf049868f253cf719b625151460b8b453"},

File diff suppressed because it is too large Load Diff

@ -8,13 +8,13 @@
- regex: 'NetFront'
name: 'NetFront'
- regex: 'Edge'
- regex: 'Edge/'
name: 'Edge'
- regex: 'Trident'
name: 'Trident'
- regex: '(?<!SmartHu)Blink'
- regex: 'Chrome/(?!1?\d\.|2[0-7]\.)'
name: 'Blink'
- regex: '(?:Apple)?WebKit'

@ -5,6 +5,82 @@
# @license http://www.gnu.org/licenses/lgpl.html LGPL v3 or later
###############
# Quick Search TV (https://play.google.com/store/apps/details?id=com.aospstudio.tvsearch)
- regex: 'Quick Search TV(?:/(?:Wild Moon Edition )?(\d+[\.\d]+))?'
name: 'Quick Search TV'
version: '$1'
# Skye (https://innatical.com/)
- regex: 'Skye/(\d+\.[\.\d]+)'
name: 'Skye'
version: '$1'
engine:
default: 'Blink'
# VD Browser (vpn.video.downloader)
- regex: 'VD/\d+'
name: 'VD Browser'
version: ''
engine:
default: 'Blink'
# SecureX (https://play.google.com/store/apps/details?id=com.securex.browser)
- regex: '\[(?:HB/29|PB/(?:66|81))\]'
name: 'SecureX'
version: ''
engine:
default: 'Blink'
# HotBrowser (https://play.google.com/store/apps/details?id=com.hotsurf.browser)
- regex: '\[HS/\d+\]'
name: 'HotBrowser'
version: ''
engine:
default: 'Blink'
# Proxy Browser (https://play.google.com/store/apps/details?id=proxy.browser.unblock.sites.proxybrowser.unblocksites)
- regex: '\[PB/\d+\]'
name: 'Proxy Browser'
version: ''
engine:
default: 'Blink'
# Onion Browser (https://apps.apple.com/us/app/onion-browser/id519296448)
- regex: '^Normalized (?:iPad|iPhone) \(iOS Safari\)'
name: 'Onion Browser'
version: ''
engine:
default: 'WebKit'
# fGet (https://apps.apple.com/us/app/fget-file-manager-browser/id1582654012)
- regex: 'fGet/'
name: 'fGet'
version: ''
# Nuviu (https://www.nuviu-browser.com/)
- regex: 'Nuviu/(?:(\d+[\.\d]+))?'
name: 'Nuviu'
version: '$1'
# DoCoMo (https://play.google.com/store/apps/details?id=com.airfind.browser)
- regex: 'DoCoMo/(?:(\d+[\.\d]+))?'
name: 'DoCoMo'
version: '$1'
# Airfind Secure Browser (https://play.google.com/store/apps/details?id=com.airfind.browser)
- regex: 'com.airfind.browser/(?:(\d+[\.\d]+))?'
name: 'Airfind Secure Browser'
version: '$1'
engine:
default: 'Blink'
# Arc (https://arc.net/)
- regex: 'ArcMobile2/.+CFNetwork'
name: 'Arc'
version: ''
engine:
default: 'WebKit'
# Nuanti Meta (http://meta.nuanti.com/)
- regex: 'Nuanti(?:Meta)?/(\d+\.[\.\d]+)'
name: 'Nuanti Meta'
@ -119,6 +195,11 @@
version: '$1'
engine:
default: 'Blink'
- regex: 'Lilo/.+CFNetwork'
name: 'Lilo'
version: ''
engine:
default: 'WebKit'
# Lexi Browser (https://play.google.com/store/apps/details?id=com.lexi.browser)
- regex: 'lexi/(\d+[\.\d]+)'
@ -302,7 +383,7 @@
name: 'Edge WebView'
version: '$1'
engine:
default: 'WebKit'
default: 'Blink'
# Mandarin (https://mandarin-browser.com/)
- regex: 'Mandarin Browser/(\d+[\.\d]+)'
@ -850,7 +931,7 @@
name: 'Firefox Reality'
version: ''
# AVG Secure Browser (https://www.avg.com/en-us/secure-browser)
# AVG Secure Browser (https://www.avg.com/en-us/secure-browser | https://play.google.com/store/apps/details?id=com.avg.android.secure.browser)
- regex: 'AVG(?:/(\d+[\.\d]+))?'
name: 'AVG Secure Browser'
version: '$1'
@ -858,6 +939,11 @@
default: 'WebKit'
versions:
28: 'Blink'
- regex: 'AT/(\d+[\.\d]+)'
name: 'AVG Secure Browser'
version: '$1'
engine:
default: 'Blink'
# START Internet Browser (https://play.google.com/store/apps/details?id=fyi.start.web.browser)
- regex: 'Start/(?:(\d+[\.\d]+))?'
@ -1111,11 +1197,16 @@
version: '$1'
engine:
default: 'WebKit'
- regex: 'Chrome/.*QwantMobile(?:/(\d+[\.\d]+))?'
name: 'Qwant Mobile'
version: '$1'
engine:
default: 'Blink'
- regex: 'QwantMobile(?:/(\d+[\.\d]+))?'
name: 'Qwant Mobile'
version: '$1'
engine:
default: 'Gecko'
default: '' # multiple (Gecko on Android, WebKit on iOS)
- regex: 'Qwant/(\d+[\.\d]+)'
name: 'Qwant Mobile'
version: '$1'
@ -1186,7 +1277,7 @@
name: 'Sailfish Browser'
version: '$1'
engine:
default: 'Gecko'
default: ''
# IceCat
- regex: 'IceCat(?:/(\d+[\.\d]+))?'
@ -1719,7 +1810,7 @@
default: '' # multi engine
#Comodo Dragon
- regex: '(?:Comodo[ _])?Dragon(?!fruit)(?:/(\d+[\.\d]+))?'
- regex: '(?:Comodo[ _])?Dragon/(\d+[\.\d]+)'
name: 'Comodo Dragon'
version: '$1'
engine:
@ -1832,7 +1923,7 @@
engine:
default: 'WebKit'
versions:
'4.5': 'Blink'
'4.2': 'Blink'
- regex: '(?:Maxthon(?:%20Browser)?|MxBrowser(?:-inhouse|-iPhone)?|MXiOS)[ /](\d+[\.\d]+)?'
name: 'Maxthon'
version: '$1'
@ -1868,7 +1959,7 @@
versions:
28: 'Blink'
- regex: 'Iron(?:/(\d+[\.\d]+))?'
- regex: 'Iron/(\d+[\.\d]+)'
name: 'Iron'
version: '$1'
engine:
@ -1976,14 +2067,14 @@
name: 'Streamy'
version: '$1'
engine:
default: 'WebKit'
default: ''
#Isivioo (Android version is Chrome based)
- regex: 'isivioo'
name: 'Isivioo'
version: ''
engine:
default: 'WebKit'
default: ''
#UC Browser
- regex: 'UC[ ]?Browser(?:[ /]?(\d+[\.\d]+))?'
@ -2032,12 +2123,12 @@
name: 'Web Explorer'
version: '$1'
engine:
default: 'WebKit'
default: 'Blink'
- regex: 'webexplorer/(\d+)'
name: 'Web Explorer'
version: '$1'
engine:
default: 'WebKit'
default: ''
# Seznam Browser
- regex: 'Chrome.+SznProhlizec/(\d+[\.\d]+)'
@ -2193,28 +2284,28 @@
name: 'Quark'
version: '$1'
engine:
default: 'WebKit'
default: 'Blink'
# Monument Browser
- regex: 'MonumentBrowser(?:/(\d+[\.\d]+))?'
name: 'Monument Browser'
version: '$1'
engine:
default: 'WebKit'
default: 'Blink'
# Blue Browser
- regex: 'BlueBrowser(?:/(\d+[\.\d]+))?'
name: 'Blue Browser'
version: '$1'
engine:
default: 'WebKit'
default: 'Blink'
# Japan Browser (https://play.google.com/store/apps/details?id=com.japan.browser.anti.blokir)
- regex: 'JAPAN Browser(?:/(\d+[\.\d]+))?'
name: 'Japan Browser'
version: '$1'
engine:
default: 'WebKit'
default: 'Blink'
- regex: 'OpenFin/(?:(\d+[\.\d]+))'
name: 'OpenFin'

@ -198,6 +198,7 @@
- regex: 'Faraday(?: v(\d+[\.\d]+))?'
name: 'Faraday'
version: '$1'
url: 'https://github.com/lostisland/faraday'
- regex: '(?:Go-http-client|Go )/?(?:(\d+[\.\d]+))?(?: package http)?'
name: 'Go-http-client'
@ -520,3 +521,48 @@
- regex: 'AppleCoreMedia/1\.0\.0'
name: 'iOS Application'
version: ''
- regex: 'cpp-httplib(?:/(\d+[\.\d]+))?'
name: 'cpp-httplib'
version: '$1'
url: 'https://github.com/yhirose/cpp-httplib'
- regex: 'Definitely-Not-Requests'
name: 'Requests'
version: ''
url: 'https://github.com/psf/requests'
- regex: 'Stealer ([\d\.]+)'
name: 'Stealer'
version: '$1'
url: 'https://github.com/hotrush/stealer/'
- regex: 'Mandrill-PHP(?:/(\d+[\.\d]+))?'
name: 'Mandrill PHP'
version: '$1'
url: 'https://bitbucket.org/mailchimp/mandrill-api-php/src/master/'
- regex: 'babashka.http-client(?:/(\d+[\.\d]+))?'
name: 'Babashka HTTP Client'
version: '$1'
url: 'https://github.com/babashka/http-client'
- regex: 'http.rb(?:/(\d+[\.\d]+))?'
name: 'http.rb'
version: '$1'
url: 'https://github.com/httprb/http'
- regex: 'node-superagent(?:/(\d+[\.\d]+))?'
name: 'superagent'
version: '$1'
url: 'https://github.com/ladjs/superagent'
- regex: 'CakePHP'
name: 'CakePHP'
version: ''
url: 'https://www.cakephp.org/'
- regex: 'request.js'
name: 'request'
version: ''
url: 'https://github.com/request/request'

@ -171,11 +171,41 @@
name: '$1'
version: '$2'
# Signal uses WhatsApp user agent
- regex: 'WhatsApp(?:/2)?$'
name: 'Signal'
version: ''
# WhatsApp Business
- regex: 'WhatsApp/([\d\.]+).+CloudAPI'
name: 'WhatsApp Business'
version: '$1'
# WhatsApp
- regex: 'WhatsApp(?:[ /]([\d\.]+))?'
- regex: 'WhatsApp(?:Electron|\-app|\-linux-app)?(?:[ /]([\d\.]+))?'
name: 'WhatsApp'
version: '$1'
# WhatsApp+2
- regex: 'YoWhatsApp2Plus(?:/(\d+[\.\d]+))?'
name: 'WhatsApp+2'
version: '$1'
# FM WhatsApp
- regex: 'YoFMWhatsApp(?:/(\d+[\.\d]+))?'
name: 'FM WhatsApp'
version: '$1'
# GBWhatsApp
- regex: '(?:Yo)?GBWhatsApp(?:/(\d+[\.\d]+))?'
name: 'GBWhatsApp'
version: '$1'
# Yo WhatsApp
- regex: 'YoYoWhatsApp(?:/(\d+[\.\d]+))?'
name: 'Yo WhatsApp'
version: '$1'
# Line
- regex: 'Line(?:[ /]([\d\.]+))'
name: 'Line'
@ -469,7 +499,7 @@
version: '$1'
# TikTok (https://apps.apple.com/app/id835599320 | https://play.google.com/store/apps/details?id=com.zhiliaoapp.musically)
- regex: '(?:TikTok[/ ]|musical_ly_|AppName/musical_ly app_version/)(\d+\.(?:[\.\d]+))?'
- regex: '(?:TikTok[/ ]|com.zhiliaoapp.musically|musical_ly_|AppName/musical_ly app_version/)(\d+\.(?:[\.\d]+))?'
name: 'TikTok'
version: '$1'
@ -1086,11 +1116,6 @@
name: 'Focus Keeper'
version: '$1'
# Quick Search TV (https://play.google.com/store/apps/details?id=com.aospstudio.tvsearch)
- regex: 'Quick Search TV/(\d+\.[\.\d]+)'
name: 'Quick Search TV'
version: '$1'
# Microsoft Store (https://www.microsoft.com/en-us/store/apps/windows)
- regex: 'WindowsStoreSDK'
name: 'Microsoft Store'
@ -1112,7 +1137,7 @@
version: '$1'
# Microsoft Bing Search (https://play.google.com/store/apps/details?id=com.microsoft.bing | https://apps.apple.com/us/app/microsoft-bing-search/id345323231)
- regex: 'BingSapphire/(\d+\.[\.\d]+)'
- regex: '(?:Bing)?Sapphire/(\d+\.[\.\d]+)'
name: 'Microsoft Bing Search'
version: '$1'
@ -1132,7 +1157,7 @@
version: '$1'
# F-Secure SAFE (https://apps.apple.com/app/f-secure-safe/id572847748)
- regex: 'FSCDCSafe (\d+\.[\.\d]+)'
- regex: 'FSCDCSafe[ /](\d+\.[\.\d]+)'
name: 'F-Secure SAFE'
version: '$1'
@ -1383,22 +1408,11 @@
name: 'Amazon Music'
version: '$1'
# Klarna (https://apps.apple.com/us/app/klarna-shop-now-pay-later/id1115120118)
- regex: 'Klarna/(\d+\.[.\d]+)'
- regex: 'Klarna/(\d+\.[.\d]+)?'
name: 'Klarna'
version: '$1'
# WhatsApp+2
- regex: 'YoWhatsApp2Plus(?:/(\d+[\.\d]+))?'
name: 'WhatsApp+2'
version: '$1'
# GBWhatsApp
- regex: 'GBWhatsApp(?:/(\d+[\.\d]+))?'
name: 'GBWhatsApp'
version: '$1'
# R (https://www.r-project.org/)
- regex: '^R/(\d+[\.\d]+)'
name: 'R'
@ -2181,11 +2195,6 @@
name: 'NPR One'
version: '$1'
# unsure what this is, but is used quite a bit
- regex: 'Trackable/([\d\.]+)'
name: 'Trackable'
version: '$1'
- regex: 'Uforia/([\d\.]+)'
name: 'Uforia'
version: '$1'
@ -2194,7 +2203,7 @@
name: 'L.A. Times'
version: ''
- regex: '(?:^NYT(?: |%20)?Audio|nytios)/([\d.]+)'
- regex: '(?:^NYT(?: |%20)?Audio(?:-iOS)?|nytios)/([\d.]+)'
name: 'The New York Times'
version: '$1'
@ -2213,3 +2222,63 @@
- regex: 'IPTV/([\d\.]+)'
name: 'IPTV'
version: '$1'
# IPTV Pro (https://play.google.com/store/apps/details?id=ru.iptvremote.android.iptv.pro)
- regex: 'IPTV Pro/([\d\.]+)'
name: 'IPTV Pro'
version: '$1'
# MetaMask (https://apps.apple.com/us/app/metamask-blockchain-wallet/id1438144202)
- regex: 'MetaMask/'
name: 'MetaMask'
version: ''
# +Simple (https://play.google.com/store/apps/details?id=com.massimple.nacion.gcba.es)
- regex: '\+Simple Browser'
name: '+Simple'
version: ''
# Bitwarden (https://bitwarden.com/)
- regex: 'Bitwarden_Mobile/([\d\.]+)'
name: 'Bitwarden'
version: '$1'
# MX Player (https://play.google.com/store/apps/details?id=com.mxtech.videoplayer.ad)
- regex: 'MXPlayer/([\d\.]+)'
name: 'MX Player'
version: '$1'
# HistoryHound
- regex: 'HistoryHound/([\d\.]+)'
name: 'HistoryHound'
version: '$1'
# Quicksilver (https://qsapp.com/)
- regex: 'Quicksilver'
name: 'Quicksilver'
version: ''
# CamScanner (https://play.google.com/store/apps/details?id=com.intsig.camscanner)
- regex: 'CamScanner/([\d\.]+)'
name: 'CamScanner'
version: '$1'
# OBS Studio (https://obsproject.com/)
- regex: 'OBS/(\d+[\.\d]+)'
name: 'OBS Studio'
version: '$1'
# XSplit Broadcaster (https://www.xsplit.com/broadcaster)
- regex: 'XSplitBroadcaster/(\d+[\.\d]+)'
name: 'XSplit Broadcaster'
version: '$1'
# Twitch Studio (https://www.twitch.tv/broadcast/studio)
- regex: 'twitch-desktop-electron-platform.*spotlight/(\d+[\.\d]+)'
name: 'Twitch Studio'
version: '$1'
# Stream Master (https://www.lovense.com/stream-master)
- regex: 'Stream Master$'
name: 'Stream Master'
version: ''

@ -88,6 +88,7 @@
'de.twokit.video.tv.cast.browser.lg': 'TV Cast'
'de.twokit.video.tv.cast.browser.samsung': 'TV Cast'
'com.cleanmaster.security': 'CM Security'
'com.antivirus.master.cmsecurity': 'CM Security'
'idm.video.free': 'IDM Video Download Manager'
'mobi.deallauncher.coupons.shopping': 'Coupons & Deals'
'com.wukongtv.wkcast.intl': 'Quick Cast'
@ -95,6 +96,11 @@
'com.fsecure.ms.kpn.veilig': 'KPN Veilig'
'com.fsecure.ms.actshield': 'ACT Shield'
'com.fsecure.ms.talktalksa': 'TalkTalk SuperSafe'
'com.fsecure.ms.dnafi': 'DNA Digiturva'
'com.fsecure.ms.teliasweden': 'Telia Trygg'
'com.fsecure.ms.upc.ch': 'UPC Internet Security'
'com.fsecure.ms.sfr': 'SFR Sécurité'
'com.fsecure.ms.sonera': 'Telia Turvapaketti'
'com.bifrostwallet.app': 'Bifrost Wallet'
'com.anydesk.anydeskandroid': 'AnyDesk Remote Desktop'
'com.google.android.youtube': 'YouTube'
@ -116,6 +122,10 @@
'com.box.video.downloader': 'BOX Video Downloader'
'com.oxoo.kinogo': 'Kinogo.ge'
'com.tradron.hdvideodownloader': 'Download Hub'
'net.daum.android.daum': 'Daum'
'com.massimple.nacion.gcba.es': '+Simple'
'com.massimple.nacion.parana.es': '+Simple'
'com.microsoft.math': 'Microsoft Math Solver'
# Vpns
'org.torproject.android': 'Orbot'
@ -125,3 +135,6 @@
'com.surfshark.vpnclient.android': 'Surfshark'
'com.omshyapps.vpn': 'Omshy VPN'
'com.kuto.vpn': 'KUTO VPN'
'com.ezt.vpn': 'EZVPN'
'com.nocardteam.nocardvpn': 'NoCard VPN'
'com.nocardteam.nocardvpn.lite': 'NoCard VPN Lite'

@ -1,5 +1,8 @@
# Browsers
'mark.via.gg': 'Via'
'mark.via.gp': 'Via'
'mark.via.gq': 'Via'
'mark.via.pm': 'Via'
'com.pure.mini.browser': 'Pure Mini Browser'
'pure.lite.browser': 'Pure Lite Browser'
'acr.browser.Hexa': 'Hexa Web Browser'
@ -95,6 +98,7 @@
'com.zeesitech.bitchutebrowser': 'Bitchute Browser'
'nova.all.video.downloader': 'Nova Video Downloader Pro'
'tukidev.pronhubbrowser.tanpavpn': 'PronHub Browser'
'com.crowbar.beaverlite': 'Frost'
'com.crowbar.beaverbrowser': 'Frost+'
'com.lenovo.browser': 'Lenovo Browser'
'com.transsion.phoenix': 'Phoenix Browser'
@ -140,6 +144,8 @@
'com.security.xvpn.z35kb.amazon': 'X-VPN'
'com.security.xvpn.z35kb.huawei': 'X-VPN'
'com.yandex.browser.lite': 'Yandex Browser Lite'
'com.yandex.browser.beta': 'Yandex Browser'
'com.yandex.browser.alpha': 'Yandex Browser'
'cz.seznam.sbrowser': 'Seznam Browser'
'com.morrisxar.nav88': 'Office Browser'
'com.rabbit.incognito.browser': 'Rabbit Private Browser'
@ -149,7 +155,7 @@
'id.browser.vivid3': 'Vivid Browser Mini'
'com.browser.yo.indian': 'Yo Browser'
'com.mercandalli.android.browser': 'G Browser'
'com.bf.browser': 'BF Browser'
'com.bf.browser': 'BXE Browser'
'com.qihoo.browser': '360 Browser'
'com.qihoo.haosou': '360 Browser'
'com.qihoo.padbrowser': '360 Browser'
@ -202,6 +208,7 @@
'com.internet.tvbrowser': 'LUJO TV Browser'
'com.logicui.tvbrowser2': 'LogicUI TV Browser'
'com.opera.browser.afin': 'Opera'
'com.opera.browser.beta': 'Opera'
'com.quark.browser': 'Quark'
'jp.co.yahoo.android.ybrowser': 'Yahoo! Japan Browser'
'com.tv.browser.open': 'Open TV Browser'
@ -225,3 +232,21 @@
'com.cavebrowser': 'Cave Browser'
'com.zordo.browser': 'Zordo Browser'
'freedom.theanarch.org.freedom': 'Freedom Browser'
'com.lechneralexander.privatebrowser': 'Private Internet Browser'
'com.airfind.browser': 'Airfind Secure Browser'
'com.securex.browser': 'SecureX'
'com.sec.android.app.sbrowser.beta': 'Samsung Browser'
'threads.thor': 'Thor'
'com.androidbull.incognito.browser': 'Incognito Browser'
'com.mosoft.godzilla': 'Godzilla Browser'
'com.oceanbrowser.mobile.android': 'Ocean Browser'
'com.qmamu.browser': 'Qmamu'
'com.techlastudio.bfbrowser': 'BF Browser'
'com.befaster.bfbrowser': 'BF Browser'
'app.nextinnovations.brokeep': 'BroKeep Browser'
'org.lilo.app': 'Lilo'
'proxy.browser.unblock.sites.proxybrowser.unblocksites': 'Proxy Browser'
'com.hotsurf.browser': 'HotBrowser'
'vpn.video.downloader': 'VD Browser'
'com.aospstudio.tvsearch': 'Quick Search TV'
'com.go.browser': 'GO Browser'

@ -5,6 +5,22 @@
# @license http://www.gnu.org/licenses/lgpl.html LGPL v3 or later
###############
# BMW (https://www.bmw.com/)
BMW:
regex: 'AFTLBT962E2(?:[);/ ]|$)'
device: 'car browser'
models:
- regex: 'AFTLBT962E2(?:[);/ ]|$)'
model: 'Car (2022)'
# Jeep (https://www.jeep.com/)
Jeep:
regex: 'AFTLFT962X3(?:[);/ ]|$)'
device: 'car browser'
models:
- regex: 'AFTLFT962X3(?:[);/ ]|$)'
model: 'Wagoneer'
# Tesla Model S
Tesla:
regex: '(?:Tesla/(?:(?:develop|feature|terminal-das-fsd-eap)-)?[0-9.]+|QtCarBrowser)'

@ -46,7 +46,7 @@ JXD:
model: 'S601 WiFi'
Sony:
regex: '(?:PlayStation ?(2|3|4 Pro|4|5|Portable|Vita)|sony_tv;ps5;|\(PS3\))'
regex: '(?:PlayStation ?(4 Pro|[2-5]|Portable|Vita)|sony_tv;ps5;|\(PS3\))'
device: 'console'
models:
- regex: 'sony_tv;ps5;'
@ -55,7 +55,7 @@ Sony:
model: 'PlayStation 4 Pro'
- regex: '\(PS3\)'
model: 'PlayStation 3'
- regex: 'PlayStation ?(2|3|4 Pro|4|5|Portable|Vita)'
- regex: 'PlayStation ?(4 Pro|[2-5]|Portable|Vita)'
model: 'PlayStation $1'
# Retroid Pocket (www.goretroid.com)

File diff suppressed because it is too large Load Diff

@ -21,7 +21,7 @@ Apple:
model: 'iPod Touch 5'
- regex: '(?:Apple-)?iPod7[C,_]?1'
model: 'iPod Touch 6'
- regex: '(?:Apple-)?iPod9[C,_]?1'
- regex: '(?:Apple-)?iPod9[C,_]?1|iPodTouch7'
model: 'iPod Touch 7'
- regex: '(?:Apple-)?iPod'
model: 'iPod Touch'

@ -952,9 +952,11 @@ Silva Schneider:
# Skyworth
Skyworth:
regex: 'Sky_worth'
regex: 'Sky_worth|SWTV[;,]'
device: 'tv'
models:
- regex: 'SKWE20E21'
model: 'SKWE20E21'
- regex: 'Sky_worth;([^);/]+)'
model: '$1'
@ -986,14 +988,6 @@ Sony:
- regex: '(K[DM]L?-?[0-9]{2}[A-Z]{1,2}[0-9]{1,5})'
model: '$1'
# SWTV
SWTV:
regex: 'SWTV[;,]'
device: 'tv'
models:
- regex: 'SKWE20E21'
model: 'SKWE20E21'
# TD Systems
'TD Systems':
regex: 'TDSystems[;,]'

@ -5,6 +5,15 @@
# @license http://www.gnu.org/licenses/lgpl.html LGPL v3 or later
###############
# BrightSignOS (https://www.brightsign.biz/ecosystem/brightsign-os/)
##########
- regex: 'BrightSign/(?:[A-Z0-9]+)/(\d+[\.\d]+)'
name: 'BrightSignOS'
version: '$1'
- regex: 'BrightSign/(\d+[\.\d]+)'
name: 'BrightSignOS'
version: '$1'
##########
# LuneOS (https://www.webos-ports.org/)
##########
@ -426,7 +435,7 @@
name: 'Android'
version: '$1'
- regex: '(?:(?:Orca-)?(?<!like |/|RadioPublic |Anghami )Android|Adr|AOSP)[ /]?(?:[a-z]+ )?(\d+[\.\d]*)'
- regex: '(?:(?:Orca-)?(?<!like |/|RadioPublic |Anghami |Callpod Keeper for )Android|Adr|AOSP)[ /]?(?:[a-z]+ )?(\d+[\.\d]*)'
name: 'Android'
version: '$1'
@ -450,7 +459,7 @@
name: 'Android'
version: '$1'
- regex: 'BeyondPod|AntennaPod|Podkicker|DoggCatcher|Player FM|okhttp|Podcatcher Deluxe|Sonos/.+\(ACR_|Linux x86_64; Quest'
- regex: 'BeyondPod|AntennaPod|Podkicker|DoggCatcher|Player FM|okhttp|Podcatcher Deluxe|Sonos/.+\(ACR_|Linux x86_64; Quest|.*WhatsApp/.*A$'
name: 'Android'
version: ''
@ -714,7 +723,7 @@
name: 'Windows'
version: '$1'
- regex: 'mingw32|winhttp'
- regex: 'mingw32|winhttp|WhatsApp/.*W$'
name: 'Windows'
version: ''
@ -833,7 +842,7 @@
name: 'iPadOS'
version: '$1.$2'
- regex: 'iPad(?:; (?:iOS|iPadOS|iPhone OS)|.+CPU OS) ((1[3-7])+(?:[_\.]\d+)*)'
- regex: 'iPad(?:; (?:iOS|iPadOS|iPhone OS)|.+CPU (?:iPad |iPhone )?OS) ((1[3-7])+(?:[_\.]\d+)*)'
name: 'iPadOS'
version: '$1'
@ -868,9 +877,11 @@
name: 'iOS'
version: '$1'
- regex: '^(?!com.apple.Safari.SearchHelper|Safari).*CFNetwork/.+ Darwin/(\d+[\.\d]+)(?!.*(?:x86_64|i386|PowerMac|Power%20Macintosh))'
- regex: '^(?!com.apple.Safari.SearchHelper|Safari).*(?:CFNetwork|Mana)/.+ Darwin/(\d+[\.\d]+)(?!.*(?:x86_64|i386|PowerMac|Power%20Macintosh))'
name: 'iOS'
versions:
- regex: 'Darwin/23.4.0'
version: '17.4'
- regex: 'Darwin/23.3.0'
version: '17.3'
- regex: 'Darwin/23.2.0'
@ -1074,7 +1085,7 @@
name: 'iOS'
version: '$1'
- regex: '(?:Podcasts/(?:[\d\.]+)|Instacast(?:HD)?/(?:\d\.[\d\.abc]+)|Pocket Casts, iOS|\(iOS\)|iOS; Opera|Overcast|Castro|Podcat|iCatcher|RSSRadio/|MobileSafari/)(?!.*x86_64)'
- regex: '(?:Podcasts/(?:[\d\.]+)|Instacast(?:HD)?/(?:\d\.[\d\.abc]+)|Pocket Casts, iOS|\(iOS\)|iOS; Opera|Overcast|Castro|Podcat|iCatcher|RSSRadio/|MobileSafari/|WhatsApp/.*i$)(?!.*x86_64)'
name: 'iOS'
version: ''
@ -1097,9 +1108,11 @@
##########
# Mac
##########
- regex: '(?:CFNetwork|StudioDisplay)/.+Darwin(?:/|; )(?:[\d\.]+).+(?:x86_64|i386|Power%20Macintosh)|(?:x86_64-apple-)?darwin(?:[\d\.]+)|PowerMac|com.apple.Safari.SearchHelper|^Safari'
- regex: '(?:CFNetwork|Mana|StudioDisplay)/.+Darwin(?:/|; )(?:[\d\.]+).+(?:x86_64|i386|Power%20Macintosh)|(?:x86_64-apple-)?darwin(?:[\d\.]+)|PowerMac|com.apple.Safari.SearchHelper|^Safari'
name: 'Mac'
versions:
- regex: '(?:x86_64-apple-)?Darwin(?:/|; )?23.4.0'
version: '14.4'
- regex: '(?:x86_64-apple-)?Darwin(?:/|; )?23.3.0'
version: '14.3'
- regex: '(?:x86_64-apple-)?Darwin(?:/|; )?23.2.0'
@ -1293,7 +1306,7 @@
name: 'Mac'
version: '$1'
- regex: 'Darwin|Macintosh|Mac_PowerPC|PPC|Mac PowerPC|iMac|MacBook|macOS|AppleExchangeWebServices|com.apple.trustd|Sonos/.+\(MDCR_'
- regex: 'Darwin|Macintosh|Mac[ _]PowerPC|PPC|iMac|MacBook|macOS|AppleExchangeWebServices|com.apple.trustd|Sonos/.+\(MDCR_|WhatsApp/.*N$'
name: 'Mac'
version: ''

@ -229,6 +229,24 @@
- "ZR"
- "D6"
- "F6"
- "RC"
- "WD"
- "P3"
- "FT"
- "A9"
- "X2"
- "N3"
- "GD"
- "O9"
- "Q3"
- "F7"
- "K2"
- "P5"
- "H5"
- "V3"
- "K3"
- "Q4"
- "G2"
- ""
- "Firefox":
- "AX"
@ -261,7 +279,6 @@
- "PI"
- "PX"
- "QA"
- "QM"
- "S5"
- "SX"
- "TF"

@ -33,6 +33,7 @@
- "AB": "ABrowse"
- "BW": "AdBlock Browser"
- "A7": "Adult Browser"
- "A9": "Airfind Secure Browser"
- "AF": "ANT Fresco"
- "AG": "ANTGalio"
- "AL": "Aloha Browser"
@ -53,6 +54,7 @@
- "AV": "Amiga Voyager"
- "AW": "Amiga Aweb"
- "PN": "APN Browser"
- "RA": "Arc"
- "AI": "Arvin"
- "AK": "Ask.com"
- "AU": "Asus Browser"
@ -65,6 +67,7 @@
- "A1": "AwoX"
- "BA": "Beaker Browser"
- "BM": "Beamrise"
- "F7": "BF Browser"
- "BB": "BlackBerry Browser"
- "H1": "BrowseHere"
- "B8": "Browser Hup Pro"
@ -87,6 +90,7 @@
- "BN": "Borealis Navigator"
- "BR": "Brave"
- "BK": "BriskBard"
- "K2": "BroKeep Browser"
- "B3": "Browspeed Browser"
- "BX": "BrowseX"
- "BZ": "Browzar"
@ -94,7 +98,7 @@
- "4B": "BrowsBit"
- "BY": "Biyubi"
- "BF": "Byffox"
- "B4": "BF Browser"
- "B4": "BXE Browser"
- "CA": "Camino"
- "0C": "Cave Browser"
- "CL": "CCleaner"
@ -129,6 +133,7 @@
- "C5": "Chromium GOST"
- "CY": "Cyberfox"
- "CS": "Cheshire"
- "RC": "Crow Browser"
- "CT": "Crusta"
- "CG": "Craving Explorer"
- "CZ": "Crazy Browser"
@ -145,6 +150,7 @@
- "DT": "Delta Browser"
- "D0": "Desi Browser"
- "DS": "DeskBrowse"
- "D2": "DoCoMo"
- "DF": "Dolphin"
- "DZ": "Dolphin Zero"
- "DO": "Dorado"
@ -167,6 +173,7 @@
- "EP": "GNOME Web"
- "G1": "G Browser"
- "ES": "Espial TV Browser"
- "FG": "fGet"
- "FA": "Falkon"
- "FX": "Faux Browser"
- "F4": "Fiery Browser"
@ -192,6 +199,7 @@
- "F5": "Flyperlink"
- "FU": "FreeU"
- "F6": "Freedom Browser"
- "FT": "Frost"
- "F3": "Frost+"
- "FI": "Fulldive"
- "GA": "Galeon"
@ -199,10 +207,12 @@
- "GH": "Ghostery Privacy Browser"
- "GI": "GinxDroid Browser"
- "GB": "Glass Browser"
- "GD": "Godzilla Browser"
- "GE": "Google Earth"
- "GP": "Google Earth Pro"
- "GO": "GOG Galaxy"
- "GR": "GoBrowser"
- "G2": "GO Browser"
- "HB": "Harman Browser"
- "HS": "HasBrowser"
- "HA": "Hawk Turbo Browser"
@ -212,6 +222,7 @@
- "HI": "Hi Browser"
- "HO": "hola! Browser"
- "H4": "Holla Web Browser"
- "H5": "HotBrowser"
- "HJ": "HotJava"
- "HT": "HTC Browser"
- "HU": "Huawei Browser Mobile"
@ -230,6 +241,7 @@
- "IV": "Isivioo"
- "I8": "IVVI Browser"
- "IW": "Iceweasel"
- "N3": "Incognito Browser"
- "IN": "Inspect Browser"
- "I9": "Insta Browser"
- "IE": "Internet Explorer"
@ -321,6 +333,8 @@
- "WR": "NextWord Browser"
- "NT": "NTENT Browser"
- "NU": "Nuanti Meta"
- "NI": "Nuviu"
- "O9": "Ocean Browser"
- "OC": "Oculus Browser"
- "O6": "Odd Browser"
- "O1": "Opera Mini iOS"
@ -333,6 +347,7 @@
- "O5": "Office Browser"
- "HH": "OhHai Browser"
- "OE": "ONE Browser"
- "N4": "Onion Browser"
- "Y1": "Opera Crypto"
- "OX": "Opera GX"
- "OG": "Opera Neon"
@ -380,12 +395,16 @@
- "LY": "PolyBrowser"
- "PI": "PrivacyWall"
- "P4": "Privacy Explorer Fast Safe"
- "P3": "Private Internet Browser"
- "P5": "Proxy Browser"
- "P2": "Pi Browser"
- "P0": "PronHub Browser"
- "PC": "PSI Secure Browser"
- "RW": "Reqwireless WebViewer"
- "PS": "Microsoft Edge"
- "QA": "Qazweb"
- "Q3": "Qmamu"
- "Q4": "Quick Search TV"
- "Q2": "QQ Browser Lite"
- "Q1": "QQ Browser Mini"
- "QQ": "QQ Browser"
@ -422,6 +441,7 @@
- "S1": "SimpleBrowser"
- "3S": "SilverMob US"
- "SY": "Sizzy"
- "K3": "Skye"
- "SK": "Skyfire"
- "SS": "Seraphic Sraf"
- "KK": "SiteKiosk"
@ -430,6 +450,7 @@
- "S7": "SP Browser"
- "9S": "Sony Small Browser"
- "8S": "Secure Private Browser"
- "X2": "SecureX"
- "T1": "Stampy Browser"
- "7S": "7Star"
- "SQ": "Smart Browser"
@ -460,6 +481,7 @@
- "TR": "T-Browser"
- "TO": "t-online.de Browser"
- "TA": "Tao Browser"
- "TH": "Thor"
- "1T": "Tor Browser"
- "TF": "TenFourFox"
- "TB": "Tenta Browser"
@ -483,7 +505,9 @@
- "UE": "Ume Browser"
- "V0": "vBrowser"
- "VA": "Vast Browser"
- "V3": "VD Browser"
- "VE": "Venus Browser"
- "WD": "Vewd Browser"
- "N0": "Nova Video Downloader Pro"
- "VS": "Viasat Browser"
- "VI": "Vivaldi"

@ -1,2 +1,6 @@
- "Chrome":
- "Google Chrome"
- "Vewd Browser":
- "Vewd Core"
- "DuckDuckGo Privacy Browser":
- "DuckDuckGo"

@ -136,3 +136,21 @@
- "ZR"
- "D6"
- "F6"
- "P3"
- "FT"
- "A9"
- "X2"
- "NI"
- "FG"
- "TH"
- "N3"
- "GD"
- "O9"
- "Q3"
- "F7"
- "K2"
- "N4"
- "P5"
- "H5"
- "V3"
- "G2"

@ -111,6 +111,7 @@
- "ORA"
- "VID"
- "TIV"
- "BSN"
- ""
- "Mac":
- "MAC"

@ -12,6 +12,7 @@
- "QNX": "BlackBerry Tablet OS"
- "BOS": "Bliss OS"
- "BMP": "Brew"
- "BSN": "BrightSignOS"
- "CAI": "Caixa Mágica"
- "CES": "CentOS"
- "CST": "CentOS Stream"

@ -91,6 +91,22 @@ defmodule Plausible.DataMigration.SiteImportsTest do
assert site_import.source == :universal_analytics
end
test "removes site import when there are no stats" do
site =
insert(:site)
|> Site.start_import(~D[2021-01-02], ~D[2020-02-02], "Google Analytics", "ok")
|> Repo.update!()
_another_site_import = insert(:site_import, site: site)
assert capture_io(fn ->
assert :ok = SiteImports.run(dry_run?: false)
end) =~ "Processing 1 site"
site = Repo.reload!(site)
assert [] = Imported.list_all_imports(site)
end
test "leaves site and imports unchanged if everything fits" do
site =
insert(:site)

@ -92,6 +92,24 @@ defmodule Plausible.GoalsTest do
assert [currency: {"is invalid", _}] = changeset.errors
end
@tag :full_build_only
test "list_revenue_goals/1 lists event_names and currencies for each revenue goal" do
site = insert(:site)
Goals.create(site, %{"event_name" => "One", "currency" => "EUR"})
Goals.create(site, %{"event_name" => "Two", "currency" => "EUR"})
Goals.create(site, %{"event_name" => "Three", "currency" => "USD"})
Goals.create(site, %{"event_name" => "Four"})
Goals.create(site, %{"page_path" => "/some-page"})
revenue_goals = Goals.list_revenue_goals(site)
assert length(revenue_goals) == 3
assert %{event_name: "One", currency: :EUR} in revenue_goals
assert %{event_name: "Two", currency: :EUR} in revenue_goals
assert %{event_name: "Three", currency: :USD} in revenue_goals
end
test "create/2 clears currency for pageview goals" do
site = insert(:site)
{:ok, goal} = Goals.create(site, %{"page_path" => "/purchase", "currency" => "EUR"})

@ -43,7 +43,7 @@ defmodule Plausible.Imported.GoogleAnalytics4Test do
token_expires_at: DateTime.to_iso8601(past)
)
site_import = Plausible.Imported.get_import(job.args.import_id)
site_import = Plausible.Imported.get_import(site, job.args.import_id)
assert site_import.label == "properties/123456"

@ -0,0 +1,164 @@
defmodule Plausible.Stats.TableDeciderTest do
use Plausible.DataCase, async: true
alias Plausible.Stats.Query
import Plausible.Stats.TableDecider
test "events_join_sessions? with experimental_reduced_joins disabled" do
assert not events_join_sessions?(make_query(false, %{}))
assert not events_join_sessions?(make_query(false, %{name: "pageview"}))
assert events_join_sessions?(make_query(false, %{source: "Google"}))
assert events_join_sessions?(make_query(false, %{entry_page: "/"}))
assert events_join_sessions?(make_query(false, %{exit_page: "/"}))
end
test "events_join_sessions? with experimental_reduced_joins enabled" do
assert not events_join_sessions?(make_query(true, %{}))
assert not events_join_sessions?(make_query(true, %{name: "pageview"}))
assert not events_join_sessions?(make_query(true, %{source: "Google"}))
assert events_join_sessions?(make_query(true, %{entry_page: "/"}))
assert events_join_sessions?(make_query(true, %{exit_page: "/"}))
end
describe "partition_metrics" do
test "with no metrics or filters" do
query = make_query(false, %{})
assert partition_metrics([], query) == {[], [], []}
end
test "session-only metrics accordingly" do
query = make_query(false, %{})
assert partition_metrics([:bounce_rate, :views_per_visit], query) ==
{[], [:bounce_rate, :views_per_visit], []}
end
test "event-only metrics accordingly" do
query = make_query(false, %{})
assert partition_metrics([:total_revenue, :visitors], query) ==
{[:total_revenue, :visitors], [], []}
end
test "filters from both, event-only metrics" do
query = make_query(false, %{name: "pageview", source: "Google"})
assert partition_metrics([:total_revenue], query) == {[:total_revenue], [], []}
end
test "filters from both, session-only metrics" do
query = make_query(false, %{name: "pageview", source: "Google"})
assert partition_metrics([:bounce_rate], query) == {[], [:bounce_rate], []}
end
test "session filters but no session metrics" do
query = make_query(false, %{source: "Google"})
assert partition_metrics([:total_revenue], query) == {[:total_revenue], [], []}
end
test "sample_percent is added to both types of metrics" do
query = make_query(false, %{})
assert partition_metrics([:total_revenue, :sample_percent], query) ==
{[:total_revenue, :sample_percent], [], []}
assert partition_metrics([:bounce_rate, :sample_percent], query) ==
{[], [:bounce_rate, :sample_percent], []}
assert partition_metrics([:total_revenue, :bounce_rate, :sample_percent], query) ==
{[:total_revenue, :sample_percent], [:bounce_rate, :sample_percent], []}
end
test "other metrics put in its own result" do
query = make_query(false, %{})
assert partition_metrics([:time_on_page, :percentage, :total_visitors], query) ==
{[], [], [:time_on_page, :percentage, :total_visitors]}
end
test "raises if unknown metric" do
query = make_query(false, %{})
assert_raise ArgumentError, fn ->
partition_metrics([:foobar], query)
end
end
end
describe "partition_metrics with experimental_reduced_joins enabled" do
test "metrics that can be calculated on either when event-only metrics" do
query = make_query(true, %{})
assert partition_metrics([:total_revenue, :visitors], query) ==
{[:total_revenue, :visitors], [], []}
assert partition_metrics([:pageviews, :visits], query) == {[:pageviews, :visits], [], []}
end
test "metrics that can be calculated on either when session-only metrics" do
query = make_query(true, %{})
assert partition_metrics([:bounce_rate, :visitors], query) ==
{[], [:bounce_rate, :visitors], []}
assert partition_metrics([:visit_duration, :visits], query) ==
{[], [:visit_duration, :visits], []}
end
test "metrics that can be calculated on either are biased to sessions" do
query = make_query(true, %{})
assert partition_metrics([:bounce_rate, :total_revenue, :visitors], query) ==
{[:total_revenue], [:bounce_rate, :visitors], []}
end
test "sample_percent is handled with either metrics" do
query = make_query(true, %{})
assert partition_metrics([:visitors, :sample_percent], query) ==
{[], [:visitors, :sample_percent], []}
end
test "metric can be calculated on either, but filtering on events" do
query = make_query(true, %{name: "pageview"})
assert partition_metrics([:visitors], query) == {[:visitors], [], []}
end
test "metric can be calculated on either, but filtering on events and sessions" do
query = make_query(true, %{name: "pageview", exit_page: "/"})
assert partition_metrics([:visitors], query) == {[], [:visitors], []}
end
test "metric can be calculated on either, filtering on either" do
query = make_query(true, %{source: "Google"})
assert partition_metrics([:visitors], query) == {[], [:visitors], []}
end
test "metric can be calculated on either, filtering on sessions" do
query = make_query(true, %{exit_page: "/"})
assert partition_metrics([:visitors], query) == {[], [:visitors], []}
end
test "breakdown value leans metric" do
query = make_query(true, %{})
assert partition_metrics([:visitors], query, "event:name") == {[:visitors], [], []}
assert partition_metrics([:visitors], query, "visit:source") == {[], [:visitors], []}
assert partition_metrics([:visitors], query, "visit:exit_page") == {[], [:visitors], []}
end
end
defp make_query(experimental_reduced_joins?, filters) do
Query.from(build(:site), %{
"experimental_reduced_joins" => to_string(experimental_reduced_joins?),
"filters" => Jason.encode!(filters)
})
end
end

@ -1355,50 +1355,14 @@ defmodule PlausibleWeb.Api.ExternalStatsController.AggregateTest do
assert json_response(conn, 200)["results"] == %{"visitors" => %{"value" => 3}}
end
test "joins correctly with the sessions (CollapsingMergeTree) table", %{
test "handles filtering by visit country", %{
conn: conn,
site: site
} do
create_sessions([
%{
site_id: site.id,
session_id: 1000,
country_code: "EE",
sign: 1,
events: 1
},
%{
site_id: site.id,
session_id: 1000,
country_code: "EE",
sign: -1,
events: 1
},
%{
site_id: site.id,
session_id: 1000,
country_code: "EE",
sign: 1,
events: 2
}
])
create_events([
%{
site_id: site.id,
session_id: 1000,
name: "pageview"
},
%{
site_id: site.id,
session_id: 1000,
name: "pageview"
},
%{
site_id: site.id,
session_id: 1000,
name: "pageview"
}
populate_stats(site, [
build(:pageview, country_code: "EE"),
build(:pageview, country_code: "EE"),
build(:pageview, country_code: "EE")
])
conn =

@ -1384,58 +1384,7 @@ defmodule PlausibleWeb.Api.StatsController.PagesTest do
]
end
test "returns top entry pages by visitors filtered by hostname", %{conn: conn, site: site} do
populate_stats(site, [
build(:pageview,
pathname: "/page1",
hostname: "en.example.com",
timestamp: ~N[2021-01-01 00:00:00]
),
build(:pageview,
pathname: "/page1",
hostname: "es.example.com",
timestamp: ~N[2021-01-01 00:00:00]
),
build(:pageview,
pathname: "/page2",
hostname: "en.example.com",
user_id: @user_id,
timestamp: ~N[2021-01-01 00:00:00]
),
build(:pageview,
pathname: "/page2",
hostname: "es.example.com",
user_id: @user_id,
timestamp: ~N[2021-01-01 00:15:00]
),
build(:pageview,
pathname: "/exit",
hostname: "es.example.com",
user_id: @user_id,
timestamp: ~N[2021-01-01 00:16:00]
),
build(:pageview,
pathname: "/page2",
hostname: "es.example.com",
timestamp: ~N[2021-01-01 23:15:00]
)
])
filters = Jason.encode!(%{"hostname" => "es.example.com"})
conn =
get(
conn,
"/api/stats/#{site.domain}/entry-pages?period=day&date=2021-01-01&filters=#{filters}"
)
assert json_response(conn, 200) == [
%{"name" => "/page2", "visit_duration" => 480, "visitors" => 2, "visits" => 2},
%{"name" => "/page1", "visit_duration" => 0, "visitors" => 1, "visits" => 1}
]
end
test "returns top entry pages by visitors filtered by hostname with experimental_hostname_filter",
test "returns top entry pages by visitors filtered by hostname",
%{conn: conn, site: site} do
populate_stats(site, [
build(:pageview,
@ -1478,7 +1427,7 @@ defmodule PlausibleWeb.Api.StatsController.PagesTest do
conn =
get(
conn,
"/api/stats/#{site.domain}/entry-pages?period=day&date=2021-01-01&filters=#{filters}&experimental_hostname_filter=true"
"/api/stats/#{site.domain}/entry-pages?period=day&date=2021-01-01&filters=#{filters}"
)
# We're going to only join sessions where the exit hostname matches the filter
@ -1638,54 +1587,7 @@ defmodule PlausibleWeb.Api.StatsController.PagesTest do
]
end
test "returns top exit pages by visitors filtered by hostname", %{conn: conn, site: site} do
populate_stats(site, [
build(:pageview,
pathname: "/page1",
hostname: "en.example.com",
timestamp: ~N[2021-01-01 00:00:00]
),
build(:pageview,
pathname: "/page1",
hostname: "es.example.com",
timestamp: ~N[2021-01-01 00:00:00]
),
build(:pageview,
pathname: "/page1",
hostname: "en.example.com",
user_id: @user_id,
timestamp: ~N[2021-01-01 00:00:00]
),
build(:pageview,
pathname: "/page2",
hostname: "es.example.com",
user_id: @user_id,
timestamp: ~N[2021-01-01 00:15:00]
),
build(:pageview,
pathname: "/exit",
hostname: "en.example.com",
user_id: @user_id,
timestamp: ~N[2021-01-01 00:16:00]
)
])
filters = Jason.encode!(%{hostname: "es.example.com"})
conn =
get(
conn,
"/api/stats/#{site.domain}/exit-pages?period=day&date=2021-01-01&filters=#{filters}"
)
assert json_response(conn, 200) ==
[
%{"name" => "/exit", "visitors" => 1, "visits" => 1},
%{"name" => "/page1", "visitors" => 1, "visits" => 1}
]
end
test "returns top exit pages by visitors filtered by hostname with experimental_hostname_filter",
test "returns top exit pages by visitors filtered by hostname",
%{conn: conn, site: site} do
populate_stats(site, [
build(:pageview,
@ -1723,7 +1625,7 @@ defmodule PlausibleWeb.Api.StatsController.PagesTest do
conn =
get(
conn,
"/api/stats/#{site.domain}/exit-pages?period=day&date=2021-01-01&filters=#{filters}&experimental_hostname_filter=true"
"/api/stats/#{site.domain}/exit-pages?period=day&date=2021-01-01&filters=#{filters}"
)
# We're going to only join sessions where the entry hostname matches the filter

@ -1221,49 +1221,7 @@ defmodule PlausibleWeb.Api.StatsController.SourcesTest do
]
end
test "returns top referrers for a custom goal and filtered by hostname", %{
conn: conn,
site: site
} do
populate_stats(site, [
build(:pageview,
hostname: "blog.example.com",
referrer_source: "Facebook",
user_id: @user_id
),
build(:pageview,
hostname: "app.example.com",
pathname: "/register",
user_id: @user_id
),
build(:event,
name: "Signup",
hostname: "app.example.com",
pathname: "/register",
user_id: @user_id
)
])
filters = Jason.encode!(%{goal: "Signup", hostname: "app.example.com"})
conn =
get(
conn,
"/api/stats/#{site.domain}/sources?period=day&filters=#{filters}"
)
assert json_response(conn, 200) ==
[
%{
"conversion_rate" => 100.0,
"name" => "Facebook",
"total_visitors" => 1,
"visitors" => 1
}
]
end
test "returns no top referrers for a custom goal and filtered by hostname and experimental_hostname_filter",
test "returns no top referrers for a custom goal and filtered by hostname",
%{
conn: conn,
site: site
@ -1292,13 +1250,13 @@ defmodule PlausibleWeb.Api.StatsController.SourcesTest do
conn =
get(
conn,
"/api/stats/#{site.domain}/sources?period=day&filters=#{filters}&experimental_hostname_filter=true"
"/api/stats/#{site.domain}/sources?period=day&filters=#{filters}"
)
assert json_response(conn, 200) == []
end
test "returns top referrers for a custom goal and filtered by hostname and experimental_hostname_filter",
test "returns top referrers for a custom goal and filtered by hostname (2)",
%{
conn: conn,
site: site
@ -1323,7 +1281,7 @@ defmodule PlausibleWeb.Api.StatsController.SourcesTest do
conn =
get(
conn,
"/api/stats/#{site.domain}/sources?period=day&filters=#{filters}&experimental_hostname_filter=true"
"/api/stats/#{site.domain}/sources?period=day&filters=#{filters}"
)
assert json_response(conn, 200) == [

@ -6,6 +6,36 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
describe "GET /api/stats/top-stats - default" do
setup [:create_user, :log_in, :create_new_site]
test "returns graph_metric key for graphable top stats", %{conn: conn, site: site} do
[visitors, visits, pageviews, views_per_visit, bounce_rate, visit_duration] =
conn
|> get("/api/stats/#{site.domain}/top-stats")
|> json_response(200)
|> Map.get("top_stats")
assert %{"graph_metric" => "visitors"} = visitors
assert %{"graph_metric" => "visits"} = visits
assert %{"graph_metric" => "pageviews"} = pageviews
assert %{"graph_metric" => "views_per_visit"} = views_per_visit
assert %{"graph_metric" => "bounce_rate"} = bounce_rate
assert %{"graph_metric" => "visit_duration"} = visit_duration
end
test "returns graph_metric key for graphable top stats in realtime mode", %{
conn: conn,
site: site
} do
[current_visitors, unique_visitors, pageviews] =
conn
|> get("/api/stats/#{site.domain}/top-stats?period=realtime")
|> json_response(200)
|> Map.get("top_stats")
refute Map.has_key?(current_visitors, "graph_metric")
assert %{"graph_metric" => "visitors"} = unique_visitors
assert %{"graph_metric" => "pageviews"} = pageviews
end
test "counts distinct user ids", %{conn: conn, site: site} do
populate_stats(site, [
build(:pageview, user_id: @user_id),
@ -19,7 +49,8 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
assert %{
"name" => "Unique visitors",
"value" => 2
"value" => 2,
"graph_metric" => "visitors"
} in res["top_stats"]
end
@ -36,7 +67,8 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
assert %{
"name" => "Total pageviews",
"value" => 3
"value" => 3,
"graph_metric" => "pageviews"
} in res["top_stats"]
end
@ -59,12 +91,16 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert res["top_stats"] == [
%{"name" => "Unique visitors", "value" => 2},
%{"name" => "Total visits", "value" => 2},
%{"name" => "Total pageviews", "value" => 2},
%{"name" => "Views per visit", "value" => 2.0},
%{"name" => "Bounce rate", "value" => 0},
%{"name" => "Visit duration", "value" => 120}
%{"name" => "Unique visitors", "value" => 2, "graph_metric" => "visitors"},
%{"name" => "Total visits", "value" => 2, "graph_metric" => "visits"},
%{"name" => "Total pageviews", "value" => 2, "graph_metric" => "pageviews"},
%{
"name" => "Views per visit",
"value" => 2.0,
"graph_metric" => "views_per_visit"
},
%{"name" => "Bounce rate", "value" => 0, "graph_metric" => "bounce_rate"},
%{"name" => "Visit duration", "value" => 120, "graph_metric" => "visit_duration"}
]
end
@ -80,7 +116,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Views per visit", "value" => 1.33} in res["top_stats"]
assert %{"name" => "Views per visit", "value" => 1.33, "graph_metric" => "views_per_visit"} in res[
"top_stats"
]
end
test "calculates bounce rate", %{conn: conn, site: site} do
@ -94,7 +132,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Bounce rate", "value" => 50} in res["top_stats"]
assert %{"name" => "Bounce rate", "value" => 50, "graph_metric" => "bounce_rate"} in res[
"top_stats"
]
end
test "calculates average visit duration", %{conn: conn, site: site} do
@ -116,7 +156,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Visit duration", "value" => 450} in res["top_stats"]
assert %{"name" => "Visit duration", "value" => 450, "graph_metric" => "visit_duration"} in res[
"top_stats"
]
end
test "calculates time on page instead when filtered for page", %{conn: conn, site: site} do
@ -339,7 +381,7 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
filters = Jason.encode!(%{page: "/"})
path = "/api/stats/#{site.domain}/top-stats?period=day&date=2021-01-01&filters=#{filters}"
assert %{"name" => "Bounce rate", "value" => 0} ==
assert %{"name" => "Bounce rate", "value" => 0, "graph_metric" => "bounce_rate"} ==
conn
|> get(path)
|> json_response(200)
@ -435,9 +477,17 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Bounce rate", "value" => 0} in res["top_stats"]
assert %{"name" => "Views per visit", "value" => 0.0} in res["top_stats"]
assert %{"name" => "Visit duration", "value" => 0} in res["top_stats"]
assert %{"name" => "Bounce rate", "value" => 0, "graph_metric" => "bounce_rate"} in res[
"top_stats"
]
assert %{"name" => "Views per visit", "value" => 0.0, "graph_metric" => "views_per_visit"} in res[
"top_stats"
]
assert %{"name" => "Visit duration", "value" => 0, "graph_metric" => "visit_duration"} in res[
"top_stats"
]
end
test "merges imported data into all top stat metrics", %{
@ -468,12 +518,16 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert res["top_stats"] == [
%{"name" => "Unique visitors", "value" => 3},
%{"name" => "Total visits", "value" => 3},
%{"name" => "Total pageviews", "value" => 4},
%{"name" => "Views per visit", "value" => 1.33},
%{"name" => "Bounce rate", "value" => 33},
%{"name" => "Visit duration", "value" => 303}
%{"name" => "Unique visitors", "value" => 3, "graph_metric" => "visitors"},
%{"name" => "Total visits", "value" => 3, "graph_metric" => "visits"},
%{"name" => "Total pageviews", "value" => 4, "graph_metric" => "pageviews"},
%{
"name" => "Views per visit",
"value" => 1.33,
"graph_metric" => "views_per_visit"
},
%{"name" => "Bounce rate", "value" => 33, "graph_metric" => "bounce_rate"},
%{"name" => "Visit duration", "value" => 303, "graph_metric" => "visit_duration"}
]
end
end
@ -504,7 +558,12 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
conn = get(conn, "/api/stats/#{site.domain}/top-stats?period=realtime")
res = json_response(conn, 200)
assert %{"name" => "Unique visitors (last 30 min)", "value" => 2} in res["top_stats"]
assert %{
"name" => "Unique visitors (last 30 min)",
"value" => 2,
"graph_metric" => "visitors"
} in res["top_stats"]
end
test "shows pageviews (last 30 minutes)", %{conn: conn, site: site} do
@ -518,7 +577,10 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
conn = get(conn, "/api/stats/#{site.domain}/top-stats?period=realtime")
res = json_response(conn, 200)
assert %{"name" => "Pageviews (last 30 min)", "value" => 3} in res["top_stats"]
assert %{"name" => "Pageviews (last 30 min)", "value" => 3, "graph_metric" => "pageviews"} in res[
"top_stats"
]
end
test "shows current visitors (last 5 min) with goal filter", %{conn: conn, site: site} do
@ -554,14 +616,77 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
conn = get(conn, "/api/stats/#{site.domain}/top-stats?period=realtime&filters=#{filters}")
res = json_response(conn, 200)
assert %{"name" => "Unique conversions (last 30 min)", "value" => 2} in res["top_stats"]
assert %{"name" => "Total conversions (last 30 min)", "value" => 4} in res["top_stats"]
assert %{
"name" => "Unique conversions (last 30 min)",
"value" => 2,
"graph_metric" => "visitors"
} in res["top_stats"]
assert %{
"name" => "Total conversions (last 30 min)",
"value" => 4,
"graph_metric" => "events"
} in res["top_stats"]
end
end
describe "GET /api/stats/top-stats - filters" do
setup [:create_user, :log_in, :create_new_site]
test "returns graph_metric key for graphable top stats with a page filter", %{
conn: conn,
site: site
} do
filters = Jason.encode!(%{page: "/A"})
[visitors, visits, pageviews, bounce_rate, time_on_page] =
conn
|> get("/api/stats/#{site.domain}/top-stats?filters=#{filters}")
|> json_response(200)
|> Map.get("top_stats")
assert %{"graph_metric" => "visitors"} = visitors
assert %{"graph_metric" => "visits"} = visits
assert %{"graph_metric" => "pageviews"} = pageviews
assert %{"graph_metric" => "bounce_rate"} = bounce_rate
refute Map.has_key?(time_on_page, "graph_metric")
end
test "returns graph_metric key for graphable top stats with a goal filter", %{
conn: conn,
site: site
} do
filters = Jason.encode!(%{goal: "Signup"})
[unique_visitors, unique_conversions, total_conversions, cr] =
conn
|> get("/api/stats/#{site.domain}/top-stats?filters=#{filters}")
|> json_response(200)
|> Map.get("top_stats")
refute Map.has_key?(unique_visitors, "graph_metric")
assert %{"graph_metric" => "visitors"} = unique_conversions
assert %{"graph_metric" => "events"} = total_conversions
assert %{"graph_metric" => "conversion_rate"} = cr
end
test "returns graph_metric key for graphable top stats with a goal filter in realtime mode",
%{conn: conn, site: site} do
filters = Jason.encode!(%{goal: "Signup"})
[current_visitors, unique_conversions, total_conversions] =
conn
|> get("/api/stats/#{site.domain}/top-stats?period=realtime&filters=#{filters}")
|> json_response(200)
|> Map.get("top_stats")
refute Map.has_key?(current_visitors, "graph_metric")
assert %{"graph_metric" => "visitors"} = unique_conversions
assert %{"graph_metric" => "events"} = total_conversions
end
test "returns only visitors from a country based on alpha2 code", %{conn: conn, site: site} do
populate_stats(site, [
build(:pageview, country_code: "US"),
@ -579,7 +704,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 2} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 2, "graph_metric" => "visitors"} in res[
"top_stats"
]
end
test "page glob filter", %{conn: conn, site: site} do
@ -599,7 +726,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 2} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 2, "graph_metric" => "visitors"} in res[
"top_stats"
]
end
test "contains (~) filter", %{conn: conn, site: site} do
@ -619,7 +748,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 2} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 2, "graph_metric" => "visitors"} in res[
"top_stats"
]
end
test "returns only visitors with specific screen size", %{conn: conn, site: site} do
@ -639,7 +770,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 2} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 2, "graph_metric" => "visitors"} in res[
"top_stats"
]
end
test "returns only visitors with specific screen size for a given hostname", %{
@ -670,8 +803,13 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res =
json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 3} in res["top_stats"]
assert %{"name" => "Total visits", "value" => 3} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 3, "graph_metric" => "visitors"} in res[
"top_stats"
]
assert %{"name" => "Total visits", "value" => 3, "graph_metric" => "visits"} in res[
"top_stats"
]
end
test "returns only visitors with specific browser", %{conn: conn, site: site} do
@ -691,7 +829,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 2} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 2, "graph_metric" => "visitors"} in res[
"top_stats"
]
end
test "returns only visitors with specific operating system", %{conn: conn, site: site} do
@ -711,7 +851,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 2} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 2, "graph_metric" => "visitors"} in res[
"top_stats"
]
end
test "returns number of visits from one specific referral source", %{conn: conn, site: site} do
@ -744,7 +886,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Total visits", "value" => 2} in res["top_stats"]
assert %{"name" => "Total visits", "value" => 2, "graph_metric" => "visits"} in res[
"top_stats"
]
end
test "does not return the views_per_visit metric when a page filter is applied", %{
@ -789,8 +933,13 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 2} in res["top_stats"]
assert %{"name" => "Total pageviews", "value" => 2} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 2, "graph_metric" => "visitors"} in res[
"top_stats"
]
assert %{"name" => "Total pageviews", "value" => 2, "graph_metric" => "pageviews"} in res[
"top_stats"
]
end
test "hostname glob filter", %{conn: conn, site: site} do
@ -814,8 +963,13 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res =
json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 4} in res["top_stats"]
assert %{"name" => "Total pageviews", "value" => 6} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 4, "graph_metric" => "visitors"} in res[
"top_stats"
]
assert %{"name" => "Total pageviews", "value" => 6, "graph_metric" => "pageviews"} in res[
"top_stats"
]
end
test "hostname glob subdomain filter", %{conn: conn, site: site} do
@ -833,7 +987,6 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
])
filters = Jason.encode!(%{hostname: "*.example.com"})
# filters = Jason.encode!(%{page: "/blog/*"})
conn =
get(
@ -843,8 +996,13 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique visitors", "value" => 3} in res["top_stats"]
assert %{"name" => "Total pageviews", "value" => 4} in res["top_stats"]
assert %{"name" => "Unique visitors", "value" => 3, "graph_metric" => "visitors"} in res[
"top_stats"
]
assert %{"name" => "Total pageviews", "value" => 4, "graph_metric" => "pageviews"} in res[
"top_stats"
]
end
end
@ -890,7 +1048,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Unique conversions", "value" => 1} in res["top_stats"]
assert %{"name" => "Unique conversions", "value" => 1, "graph_metric" => "visitors"} in res[
"top_stats"
]
end
test "returns conversion rate", %{conn: conn, site: site} do
@ -911,7 +1071,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Conversion rate", "value" => 33.3} in res["top_stats"]
assert %{"name" => "Conversion rate", "value" => 33.3, "graph_metric" => "conversion_rate"} in res[
"top_stats"
]
end
test "returns conversion rate without change when comparison mode disallowed", %{
@ -935,7 +1097,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Conversion rate", "value" => 33.3} in res["top_stats"]
assert %{"name" => "Conversion rate", "value" => 33.3, "graph_metric" => "conversion_rate"} in res[
"top_stats"
]
end
@tag :full_build_only
@ -972,12 +1136,14 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
assert %{
"name" => "Average revenue",
"value" => %{"long" => "$1,659.50", "short" => "$1.7K"}
"value" => %{"long" => "$1,659.50", "short" => "$1.7K"},
"graph_metric" => "average_revenue"
} in top_stats
assert %{
"name" => "Total revenue",
"value" => %{"long" => "$3,319.00", "short" => "$3.3K"}
"value" => %{"long" => "$3,319.00", "short" => "$3.3K"},
"graph_metric" => "total_revenue"
} in top_stats
end
@ -1029,12 +1195,14 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
assert %{
"name" => "Average revenue",
"value" => %{"long" => "$1,659.50", "short" => "$1.7K"}
"value" => %{"long" => "$1,659.50", "short" => "$1.7K"},
"graph_metric" => "average_revenue"
} in top_stats
assert %{
"name" => "Total revenue",
"value" => %{"long" => "$6,638.00", "short" => "$6.6K"}
"value" => %{"long" => "$6,638.00", "short" => "$6.6K"},
"graph_metric" => "total_revenue"
} in top_stats
end
@ -1101,7 +1269,7 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
describe "GET /api/stats/top-stats - with comparisons" do
setup [:create_user, :log_in, :create_new_site, :add_imported_data]
test "defaults to previous period when comparison is not set", %{site: site, conn: conn} do
test "does not return comparisons by default", %{site: site, conn: conn} do
populate_stats(site, [
build(:pageview, timestamp: ~N[2020-12-31 00:00:00]),
build(:pageview, timestamp: ~N[2020-12-31 00:00:00]),
@ -1114,7 +1282,9 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"name" => "Total visits", "value" => 3} in res["top_stats"]
assert %{"name" => "Total visits", "value" => 3, "graph_metric" => "visits"} in res[
"top_stats"
]
end
test "returns comparison data when mode is custom", %{site: site, conn: conn} do
@ -1136,7 +1306,13 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
res = json_response(conn, 200)
assert %{"change" => 0, "comparison_value" => 3, "name" => "Total visits", "value" => 3} in res[
assert %{
"change" => 0,
"comparison_value" => 3,
"name" => "Total visits",
"value" => 3,
"graph_metric" => "visits"
} in res[
"top_stats"
]
end
@ -1195,7 +1371,13 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
assert %{"top_stats" => top_stats, "with_imported" => true} = json_response(conn, 200)
assert %{"change" => 100, "comparison_value" => 2, "name" => "Total visits", "value" => 4} in top_stats
assert %{
"change" => 100,
"comparison_value" => 2,
"name" => "Total visits",
"value" => 4,
"graph_metric" => "visits"
} in top_stats
end
test "does not compare imported data when with_imported is set to false", %{
@ -1230,7 +1412,13 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
assert %{"top_stats" => top_stats, "with_imported" => false} = json_response(conn, 200)
assert %{"change" => 100, "comparison_value" => 0, "name" => "Total visits", "value" => 4} in top_stats
assert %{
"change" => 100,
"comparison_value" => 0,
"name" => "Total visits",
"value" => 4,
"graph_metric" => "visits"
} in top_stats
end
test "compares conversion rates", %{conn: conn, site: site} do
@ -1258,7 +1446,8 @@ defmodule PlausibleWeb.Api.StatsController.TopStatsTest do
"change" => -33.4,
"comparison_value" => 66.7,
"name" => "Conversion rate",
"value" => 33.3
"value" => 33.3,
"graph_metric" => "conversion_rate"
} in res["top_stats"]
end
end

@ -139,6 +139,38 @@ defmodule PlausibleWeb.GoogleAnalyticsControllerTest do
"We were unable to authenticate your Google Analytics account"
end
test "redirects to #{view} on timeout error with flash error (legacy: #{legacy})", %{
conn: conn,
site: site
} do
expect(
Plausible.HTTPClient.Mock,
:get,
fn _url, _opts ->
{:error, %Mint.TransportError{reason: :timeout}}
end
)
conn =
conn
|> get("/#{site.domain}/import/google-analytics/property-or-view", %{
"access_token" => "token",
"refresh_token" => "foo",
"expires_at" => "2022-09-22T20:01:37.112777",
"legacy" => unquote(legacy)
})
assert redirected_to(conn, 302) ==
PlausibleWeb.Router.Helpers.site_path(
conn,
unquote(view),
site.domain
)
assert Phoenix.Flash.get(conn.assigns.flash, :error) =~
"Google Analytics API has timed out."
end
test "redirects to #{view} on list retrival failure with flash error (legacy: #{legacy})",
%{
conn: conn,
@ -460,6 +492,40 @@ defmodule PlausibleWeb.GoogleAnalyticsControllerTest do
assert Phoenix.Flash.get(conn.assigns.flash, :error) =~
"Google Analytics authentication seems to have expired."
end
test "redirects to #{view} on timeout with flash error (legacy: #{legacy})",
%{
conn: conn,
site: site
} do
expect(
Plausible.HTTPClient.Mock,
:post,
fn _url, _opts, _params ->
{:error, %Mint.TransportError{reason: :timeout}}
end
)
conn =
conn
|> post("/#{site.domain}/import/google-analytics/property-or-view", %{
"property_or_view" => "properties/428685906",
"access_token" => "token",
"refresh_token" => "foo",
"expires_at" => "2022-09-22T20:01:37.112777",
"legacy" => unquote(legacy)
})
assert redirected_to(conn, 302) ==
PlausibleWeb.Router.Helpers.site_path(
conn,
unquote(view),
site.domain
)
assert Phoenix.Flash.get(conn.assigns.flash, :error) =~
"Google Analytics API has timed out."
end
end
end
@ -624,6 +690,42 @@ defmodule PlausibleWeb.GoogleAnalyticsControllerTest do
assert Phoenix.Flash.get(conn.assigns.flash, :error) =~
"Google Analytics authentication seems to have expired."
end
test "redirects to #{view} on timeout with flash error (legacy: #{legacy})",
%{
conn: conn,
site: site
} do
expect(
Plausible.HTTPClient.Mock,
:get,
fn _url, _params ->
{:error, %Mint.TransportError{reason: :timeout}}
end
)
conn =
conn
|> get("/#{site.domain}/import/google-analytics/confirm", %{
"property_or_view" => "properties/428685906",
"access_token" => "token",
"refresh_token" => "foo",
"expires_at" => "2022-09-22T20:01:37.112777",
"start_date" => "2024-02-22",
"end_date" => "2024-02-26",
"legacy" => unquote(legacy)
})
assert redirected_to(conn, 302) ==
PlausibleWeb.Router.Helpers.site_path(
conn,
unquote(view),
site.domain
)
assert Phoenix.Flash.get(conn.assigns.flash, :error) =~
"Google Analytics API has timed out."
end
end
end
@ -753,6 +855,46 @@ defmodule PlausibleWeb.GoogleAnalyticsControllerTest do
)
end
test "does not start another import when there's any other in progress for the same site", %{
conn: conn,
site: site,
user: user
} do
{:ok, job} =
Plausible.Imported.NoopImporter.new_import(site, user,
start_date: ~D[2020-01-02],
end_date: ~D[2021-10-11]
)
post(conn, "/#{site.domain}/settings/google-import", %{
"property_or_view" => "properties/123456",
"start_date" => "2018-03-01",
"end_date" => "2022-03-01",
"access_token" => "token",
"refresh_token" => "foo",
"expires_at" => "2022-09-22T20:01:37.112777",
"legacy" => "false"
})
conn =
post(conn, "/#{site.domain}/settings/google-import", %{
"property_or_view" => "123456",
"start_date" => "2018-03-01",
"end_date" => "2022-03-01",
"access_token" => "token",
"refresh_token" => "foo",
"expires_at" => "2022-09-22T20:01:37.112777",
"legacy" => "true"
})
assert Phoenix.Flash.get(conn.assigns.flash, :error) =~
"There's another import still in progress."
assert [%{id: import_id, legacy: false}] = Plausible.Imported.list_all_imports(site)
assert job.args.import_id == import_id
end
for {legacy, view} <- [{"true", :settings_integrations}, {"false", :settings_imports_exports}] do
test "redirects to #{view} with no time window error flash error (legacy: #{legacy})", %{
conn: conn,

@ -680,6 +680,25 @@ defmodule PlausibleWeb.SiteControllerTest do
assert resp =~ "Google Analytics (123456)"
assert resp =~ "(98 page views)"
end
test "disables import buttons when there's import in progress", %{conn: conn, site: site} do
_site_import1 = insert(:site_import, site: site, status: SiteImport.completed())
_site_import2 = insert(:site_import, site: site, status: SiteImport.importing())
conn = get(conn, "/#{site.domain}/settings/imports-exports")
assert html_response(conn, 200) =~ "No new imports can be started"
end
test "enables import buttons when all imports are in completed or failed state", %{
conn: conn,
site: site
} do
_site_import1 = insert(:site_import, site: site, status: SiteImport.completed())
_site_import2 = insert(:site_import, site: site, status: SiteImport.failed())
conn = get(conn, "/#{site.domain}/settings/imports-exports")
refute html_response(conn, 200) =~ "No new imports can be started"
end
end
describe "GET /:website/settings/integrations for self-hosting" do

@ -90,31 +90,6 @@ defmodule Plausible.TestUtils do
Plausible.IngestRepo.insert_all(Plausible.ClickhouseEventV2, pageviews)
end
def create_events(events) do
events =
Enum.map(events, fn event ->
Factory.build(:event, event)
|> Map.from_struct()
|> Map.delete(:__meta__)
|> update_in([:timestamp], &to_naive_truncate/1)
end)
Plausible.IngestRepo.insert_all(Plausible.ClickhouseEventV2, events)
end
def create_sessions(sessions) do
sessions =
Enum.map(sessions, fn session ->
Factory.build(:ch_session, session)
|> Map.from_struct()
|> Map.delete(:__meta__)
|> update_in([:timestamp], &to_naive_truncate/1)
|> update_in([:start], &to_naive_truncate/1)
end)
Plausible.IngestRepo.insert_all(Plausible.ClickhouseSessionV2, sessions)
end
def log_in(%{user: user, conn: conn}) do
conn =
init_session(conn)

@ -3,6 +3,14 @@ Mox.defmock(Plausible.HTTPClient.Mock, for: Plausible.HTTPClient.Interface)
Application.ensure_all_started(:double)
FunWithFlags.enable(:imports_exports)
FunWithFlags.enable(:shield_pages)
# Temporary flag to test `experimental_reduced_joins` flag on all tests.
if System.get_env("TEST_EXPERIMENTAL_REDUCED_JOINS") == "1" do
FunWithFlags.enable(:experimental_reduced_joins)
else
FunWithFlags.disable(:experimental_reduced_joins)
end
Ecto.Adapters.SQL.Sandbox.mode(Plausible.Repo, :manual)
if Mix.env() == :small_test do

Loading…
Cancel
Save